Solr engine. More...

Inheritance diagram for search_solr\engine:

Public Member Functions
	__construct ()
	Initialises the search engine configuration. More...

	add_document ($document, $fileindexing=false)
	Adds a document to the search engine. More...

	add_documents ($iterator, $searcharea, $options)
	Loop through given iterator of search documents and and have the search engine back end add them to the index. More...

	add_fields ($query)
	Sets fields to be returned in the result. More...

	add_highlight_content ($response)
	Finds the key common to both highlighing and docs array returned from response. More...

	area_index_complete ($searcharea, $numdocs=0, $fullindex=false)
	Do any area cleanup needed, and do anything to confirm contents. More...

	area_index_starting ($searcharea, $fullindex=false)
	Do anything that may need to be done before an area is indexed. More...

	check_latest_schema ()
	Checks that the schema is the latest version. More...

	clear_query_error ()
	Clears the current query error value. More...

	delete ($areaid=null)
	Delete all area's documents. More...

	delete_by_id ($id)
	Deletes the specified document. More...

	delete_index_for_context (int $oldcontextid)
	Solr supports deleting the index for a context. More...

	delete_index_for_course (int $oldcourseid)
	Solr supports deleting the index for a course. More...

	execute_query ($filters, $accessinfo, $limit=0)
	Prepares a Solr query, applies filters and executes it returning its results. More...

	file_indexing_enabled ()
	Return true if file indexing is supported and enabled. More...

	get_connection_url ($path)
	Return a Moodle url object for the server connection. More...

	get_curl_object ()
	Returns a curl object for conntecting to solr. More...

	get_document_classname ()
	Gets the document class used by this search engine. More...

	get_plugin_name ()
	Returns the plugin name. More...

	get_query_error ()
	Returns any error reported by the search engine when executing the provided query. More...

	get_query_total_count ()
	Returns the total number of documents available for the most recently call to execute_query. More...

	get_solr_major_version ()
	Returns the solr server major version. More...

	get_supported_orders (\context $context)
	Solr supports sort by location within course contexts or below. More...

	get_user ($userid)
	Returns user data checking the internal static cache. More...

	index_complete ($numdocs=0, $fullindex=false)
	Run any post indexing operations. More...

	index_starting ($fullindex=false)
	Run any pre-indexing operations. More...

	is_installed ()
	Checks if the PHP Solr extension is available. More...

	is_server_configured ()
	Is the solr server properly configured?. More...

	is_server_ready ()
	Pings the Solr server using search_solr config. More...

	merge_highlight_field_values ($doc, $highlighteddoc)
	Adds the highlighting array values to docs array values. More...

	optimize ()
	Defragments the index. More...

	record_applied_schema_version ($version)
	Usually called by the engine; marks that the schema has been updated. More...

	skip_schema_check ($skip=true)
	Tells the search engine to skip any time-consuming checks that it might do as part of the is_server_ready function, and only carry out a basic check that it can contact the server. More...

	standarize_solr_obj (\SolrObject $obj)
	Returns a standard php array from a SolrObject instance. More...

	supports_group_filtering ()
	Solr includes group support in the execute_query function. More...

	supports_users ()
	Solr supports search by user id. More...

Static Public Member Functions
static	clear_users_cache ()
	Clears the users cache. More...

Public Attributes
int const	AUTOCOMMIT_WITHIN = 15000
	Commit documents interval (number of miliseconds).

float const	CONTEXT_BOOST = 0.5
	Boost value for matching context (in addition to course boost)

float const	COURSE_BOOST = 1
	Boost value for matching course in location-ordered searches.

string const	DATE_FORMAT = 'Y-m-d\TH:i:s\Z'
	The date format used by solr.

const	FRAG_SIZE = 510
	Highlighting fragsize. More...

const	HIGHLIGHT_END = '@@HI_E@@'
	Marker for the end of a highlight.

const	HIGHLIGHT_START = '@@HI_S@@'
	Marker for the start of a highlight.

const	QUERY_SIZE = 120
	The maximum number of results to fetch at a time.

Protected Member Functions
	add_solr_document ($doc)
	Adds a text document to the search engine. More...

	add_stored_file ($document, $storedfile)
	Adds a file to the search engine. More...

	commit ()
	Commits all pending changes. More...

	convert_file_results ($responsedoc)
	A very lightweight handler for getting information about already indexed files from a Solr response. More...

	create_user_query ($filters, $accessinfo)
	Prepares a new query object with needed limits, filters, etc. More...

	file_is_indexable ($file)
	Checks to see if a passed file is indexable. More...

	get_course ($courseid)
	Returns a course instance checking internal caching. More...

	get_indexed_files ($document, $start=0, $rows=500)
	Get the currently indexed files for a particular document, returns the total count, and a subset of files. More...

	get_missing_docs ($missingdocs)
	Retreive any missing main documents and attach provided files. More...

	get_query_response ($query)
	Takes a query and returns the response in SolrObject format. More...

	get_response_counts ($response)
	Returns count information for a provided response. More...

	get_search_area ($areaid)
	Returns a search instance of the specified area checking internal caching. More...

	get_search_client ($triggerexception=true)
	Returns the solr client instance. More...

	grouped_files_process_response ($response, $limit=0)
	Processes grouped file results into documents, with attached matching files. More...

	process_document_files ($document)
	Index files attached to the docuemnt, ensuring the index matches the current document files. More...

	process_response ($response, $limit=0, $skipaccesscheck=false)
	Filters the response on Moodle side. More...

	set_query ($query, $q)
	Prepares a new query by setting the query, start offset and rows to return. More...

	should_skip_schema_check ()
	For use by subclasses. More...

	to_document (\core_search\base $searcharea, $docdata)
	Returns a document instance prepared to be rendered. More...

	update_schema ($oldversion, $newversion)
	Requests the search engine to upgrade the schema. More...

Static Protected Member Functions
static	replace_underlines (string $str)
	Replaces underlines at edges of words in the content with spaces. More...

Protected Attributes
bool	$cacheclient = true
	True if we should reuse SolrClients, false if not.

array	$cachedareas = array()
	Internal cache.

array	$cachedcourses = array()
	Internal cache.

SolrClient	$client = null

stdClass	$config = null
	The search engine configuration.

curl	$curl = null
	Direct curl object.

array	$highlightfields = array('title', 'content', 'description1', 'description2')
	Fields that can be highlighted.

string	$pluginname = null
	Frankenstyle plugin name.

int	$processeddocs = 0
	Number of docs we have processed for the last query.

string	$queryerror = null
	Last executed query error, if there was any.

int	$skippeddocs = 0
	Number of docs that have been skipped while processing the last query.

bool	$skipschemacheck = false
	If true, should skip schema validity check when checking the search engine is ready.

int	$solrmajorversion = null
	Solr server major version.

int	$totalenginedocs = 0
	Number of total docs reported by Sorl for the last query.

Static Protected Attributes
static stdClass[]	$cachedusers = array()
	User data required to show their fullnames. More...

Detailed Description

Solr engine.

Copyright: 2015 Daniel Neis Araujo

License: http://www.gnu.org/copyleft/gpl.html GNU GPL v3 or later

Constructor & Destructor Documentation

◆ __construct()

search_solr\engine::__construct ( )

Initialises the search engine configuration.

Return values

void

Reimplemented from core_search\engine.

Member Function Documentation

◆ add_document()

search_solr\engine::add_document	(	$document,
		$fileindexing = `false`
	)

Adds a document to the search engine.

This does not commit to the search engine.

Parameters

document	$document
bool	$fileindexing	True if file indexing is to be used

Return values

bool

Reimplemented from core_search\engine.

◆ add_documents()

core_search\engine::add_documents	(	$iterator,
		$searcharea,
		$options
	)

inherited

Loop through given iterator of search documents and and have the search engine back end add them to the index.

Parameters

iterator	$iterator	the iterator of documents to index
searcharea	$searcharea	the area for the documents to index
array	$options	document indexing options

Return values

array Processed document counts

◆ add_fields()

search_solr\engine::add_fields ( $query )

Sets fields to be returned in the result.

Parameters

SolrDisMaxQuery | SolrQuery $query object.

◆ add_highlight_content()

search_solr\engine::add_highlight_content ( $response )

Finds the key common to both highlighing and docs array returned from response.

Parameters

object $response containing results.

◆ add_solr_document()

search_solr\engine::add_solr_document ( $doc )

protected

Adds a text document to the search engine.

Parameters

array $doc

Return values

bool

◆ add_stored_file()

search_solr\engine::add_stored_file	(	$document,
		$storedfile
	)

protected

Adds a file to the search engine.

Notes about Solr and Tika indexing. We do not send the mime type, only the filename. Tika has much better content type detection than Moodle, and we will have many more doc failures if we try to send mime types.

Parameters

document	$document
stored_file	$storedfile

Return values

void

◆ area_index_complete()

search_solr\engine::area_index_complete	(	$searcharea,
		$numdocs = `0`,
		$fullindex = `false`
	)

Do any area cleanup needed, and do anything to confirm contents.

Return false to prevent the search area completed time and stats from being updated.

Parameters

core_search\base	$searcharea	The search area that was complete
int	$numdocs	The number of documents that were added to the index
bool	$fullindex	True if a full index is being performed

Return values

bool	True means that data is considered indexed

Reimplemented from core_search\engine.

◆ area_index_starting()

core_search\engine::area_index_starting	(	$searcharea,
		$fullindex = `false`
	)

inherited

Do anything that may need to be done before an area is indexed.

Parameters

core_search\base	$searcharea	The search area that was complete
bool	$fullindex	True if a full index is being performed

Return values

void

◆ check_latest_schema()

core_search\engine::check_latest_schema ( )

inherited

Checks that the schema is the latest version.

If the version stored in config does not match the current, this function will attempt to upgrade the schema.

Return values

bool|string True if schema is OK, a string if user needs to take action

◆ clear_query_error()

core_search\engine::clear_query_error ( )

inherited

Clears the current query error value.

Return values

void

◆ clear_users_cache()

static core_search\engine::clear_users_cache ( )

staticinherited

Clears the users cache.

Return values

null

◆ commit()

search_solr\engine::commit ( )

protected

Commits all pending changes.

Return values

void

◆ convert_file_results()

search_solr\engine::convert_file_results ( $responsedoc )

protected

A very lightweight handler for getting information about already indexed files from a Solr response.

Parameters

SolrObject $responsedoc A Solr response document

Return values

stdClass[] An array of objects that contain the basic information for file processing.

◆ create_user_query()

search_solr\engine::create_user_query	(	$filters,
		$accessinfo
	)

protected

Prepares a new query object with needed limits, filters, etc.

Parameters

stdClass	$filters	Containing query and filters.
stdClass	$accessinfo	Information about contexts the user can access

Return values

SolrDisMaxQuery|null Query object or null if they can't get any results

◆ delete()

search_solr\engine::delete ( $areaid = null )

Delete all area's documents.

Parameters

string $areaid

Return values

void

Reimplemented from core_search\engine.

◆ delete_by_id()

search_solr\engine::delete_by_id ( $id )

Deletes the specified document.

Parameters

string $id The document id to delete

Return values

void

◆ delete_index_for_context()

search_solr\engine::delete_index_for_context ( int $oldcontextid )

Solr supports deleting the index for a context.

Parameters

int $oldcontextid Context that has been deleted

Return values

bool	True to indicate that any data was actually deleted

Exceptions

core_search

Reimplemented from core_search\engine.

◆ delete_index_for_course()

search_solr\engine::delete_index_for_course ( int $oldcourseid )

Solr supports deleting the index for a course.

Parameters

int $oldcourseid

Return values

bool	True to indicate that any data was actually deleted

Exceptions

core_search

Reimplemented from core_search\engine.

◆ execute_query()

search_solr\engine::execute_query	(	$filters,
		$accessinfo,
		$limit = `0`
	)

Prepares a Solr query, applies filters and executes it returning its results.

Exceptions

core_search

Reimplemented from core_search\engine.

◆ file_indexing_enabled()

search_solr\engine::file_indexing_enabled ( )

Return true if file indexing is supported and enabled.

False otherwise.

Return values

bool

Reimplemented from core_search\engine.

◆ file_is_indexable()

search_solr\engine::file_is_indexable ( $file )

protected

Checks to see if a passed file is indexable.

Parameters

stored_file $file The file to check

Return values

bool	True if the file can be indexed

◆ get_connection_url()

search_solr\engine::get_connection_url ( $path )

Return a Moodle url object for the server connection.

Parameters

string $path The solr path to append.

Return values

moodle_url

◆ get_course()

core_search\engine::get_course ( $courseid )

protectedinherited

Returns a course instance checking internal caching.

Parameters

int $courseid

Return values

stdClass

◆ get_curl_object()

search_solr\engine::get_curl_object ( )

Returns a curl object for conntecting to solr.

Return values

curl

◆ get_document_classname()

core_search\engine::get_document_classname ( )

inherited

Gets the document class used by this search engine.

Search engines can overwrite core_search\document with search_ENGINENAME\document class.

Looks for a document class in the current search engine namespace, falling back to core_search\document.

Publicly available because search areas do not have access to the engine details, core_search\document_factory accesses this function.

Return values

string

◆ get_indexed_files()

search_solr\engine::get_indexed_files	(	$document,
		$start = `0`,
		$rows = `500`
	)

protected

Get the currently indexed files for a particular document, returns the total count, and a subset of files.

Parameters

document	$document
int	$start	The row to start the results on. Zero indexed.
int	$rows	The number of rows to fetch

Return values

array A two element array, the first is the total number of availble results, the second is an array of documents for the current request.

◆ get_missing_docs()

search_solr\engine::get_missing_docs ( $missingdocs )

protected

Retreive any missing main documents and attach provided files.

The missingdocs array should be an array, indexed by document id, of main documents we need to retrieve. The value associated to the key should be an array of stored_files or stored file ids to attach to the result document.

Return array also indexed by document id.

Parameters

array() $missingdocs An array, indexed by document id, with arrays of files/ids to attach.

Return values

document[]

◆ get_plugin_name()

core_search\engine::get_plugin_name ( )

inherited

Returns the plugin name.

Return values

string Frankenstyle plugin name.

◆ get_query_error()

core_search\engine::get_query_error ( )

inherited

Returns any error reported by the search engine when executing the provided query.

It should be called from static::execute_query when an exception is triggered.

Return values

string

◆ get_query_response()

search_solr\engine::get_query_response ( $query )

protected

Takes a query and returns the response in SolrObject format.

Parameters

SolrQuery $query Solr query object.

Return values

SolrObject|false Response document or false on error.

◆ get_query_total_count()

search_solr\engine::get_query_total_count ( )

Returns the total number of documents available for the most recently call to execute_query.

Return values

int

Reimplemented from core_search\engine.

◆ get_response_counts()

search_solr\engine::get_response_counts ( $response )

protected

Returns count information for a provided response.

Will return 0, 0 for invalid or empty responses.

Parameters

SolrDocument $response The response document from Solr.

Return values

array A two part array. First how many response docs are in the response. Second, how many results are vailable in the engine.

◆ get_search_area()

core_search\engine::get_search_area ( $areaid )

protectedinherited

Returns a search instance of the specified area checking internal caching.

Parameters

string $areaid Area id

Return values

core_search

◆ get_search_client()

search_solr\engine::get_search_client ( $triggerexception = true )

protected

Returns the solr client instance.

We don't reuse SolrClient if we are on libcurl 7.35.0, due to a bug in that version of curl.

Exceptions

core_search

◆ get_solr_major_version()

search_solr\engine::get_solr_major_version ( )

Returns the solr server major version.

Return values

int

◆ get_supported_orders()

search_solr\engine::get_supported_orders ( \context $context )

Solr supports sort by location within course contexts or below.

Parameters

context $context Context that the user requested search from

Return values

array Array from order name => display text

Reimplemented from core_search\engine.

◆ get_user()

core_search\engine::get_user ( $userid )

inherited

Returns user data checking the internal static cache.

Including here the minimum required user information as this may grow big.

Parameters

int $userid

Return values

stdClass

◆ grouped_files_process_response()

search_solr\engine::grouped_files_process_response	(	$response,
		$limit = `0`
	)

protected

Processes grouped file results into documents, with attached matching files.

Parameters

SolrObject	$response	The response returned from solr server
int	$limit	The maximum number of results to return. 0 for all.

Return values

array Final results to be displayed.

◆ index_complete()

core_search\engine::index_complete	(	$numdocs = `0`,
		$fullindex = `false`
	)

inherited

Run any post indexing operations.

Should be overwritten if the search engine needs to do any post index cleanup.

Parameters

int	$numdocs	The number of documents that were added to the index
bool	$fullindex	True if a full index was performed

Return values

void

◆ index_starting()

core_search\engine::index_starting ( $fullindex = false )

inherited

Run any pre-indexing operations.

Should be overwritten if the search engine needs to do any pre index preparation.

Parameters

bool $fullindex True if a full index will be performed

Return values

void

◆ is_installed()

search_solr\engine::is_installed ( )

Checks if the PHP Solr extension is available.

Return values

bool

Reimplemented from core_search\engine.

◆ is_server_configured()

search_solr\engine::is_server_configured ( )

Is the solr server properly configured?.

Return values

true|string Returns true if all good or an error string.

◆ is_server_ready()

search_solr\engine::is_server_ready ( )

Pings the Solr server using search_solr config.

Return values

true|string Returns true if all good or an error string.

Reimplemented from core_search\engine.

◆ merge_highlight_field_values()

search_solr\engine::merge_highlight_field_values	(	$doc,
		$highlighteddoc
	)

Adds the highlighting array values to docs array values.

Exceptions

core_search

◆ optimize()

search_solr\engine::optimize ( )

Defragments the index.

Return values

void

Reimplemented from core_search\engine.

◆ process_document_files()

search_solr\engine::process_document_files ( $document )

protected

Index files attached to the docuemnt, ensuring the index matches the current document files.

For documents that aren't known to be new, we check the index for existing files.

New files we will add.
Existing and unchanged files we will skip.
File that are in the index but not on the document will be deleted from the index.
Files that have changed will be re-indexed.

Parameters

document $document

◆ process_response()

search_solr\engine::process_response	(	$response,
		$limit = `0`,
		$skipaccesscheck = `false`
	)

protected

Filters the response on Moodle side.

Parameters

SolrObject	$response	Solr object containing the response return from solr server.
int	$limit	The maximum number of results to return. 0 for all.
bool	$skipaccesscheck	Don't use check_access() on results. Only to be used when results have known access.

Return values

array::$results containing final results to be displayed.

◆ record_applied_schema_version()

core_search\engine::record_applied_schema_version ( $version )

inherited

Usually called by the engine; marks that the schema has been updated.

Parameters

int $version Records the schema version now applied

◆ replace_underlines()

static search_solr\engine::replace_underlines ( string $str )

staticprotected

Replaces underlines at edges of words in the content with spaces.

For example 'frogs' will become 'frogs', 'frogs and toads' will become 'frogs and toads', and 'frogs_and_toads' will be left as 'frogs_and_toads'.

The reason for this is that for italic content_to_text puts italic underlines at the start and end of the italicised phrase (not between words). Solr treats underlines as part of the word, which means that if you search for a word in italic then you can't find it.

Parameters

string $str String to replace

Return values

string Replaced string

◆ set_query()

search_solr\engine::set_query	(	$query,
		$q
	)

protected

Prepares a new query by setting the query, start offset and rows to return.

Parameters

SolrQuery	$query
object	$q	Containing query and filters.

◆ should_skip_schema_check()

core_search\engine::should_skip_schema_check ( )

protectedinherited

For use by subclasses.

The engine can call this inside is_server_ready to check whether it should skip time-consuming schema checks.

Since: Moodle 3.5

Return values

bool	True if schema checks should be skipped

◆ skip_schema_check()

core_search\engine::skip_schema_check ( $skip = true )

inherited

Tells the search engine to skip any time-consuming checks that it might do as part of the is_server_ready function, and only carry out a basic check that it can contact the server.

This setting is not remembered and applies only to the current request.

Since: Moodle 3.5

Parameters

bool $skip True to skip the checks, false to start checking again

◆ standarize_solr_obj()

search_solr\engine::standarize_solr_obj ( \SolrObject $obj )

Returns a standard php array from a SolrObject instance.

Parameters

SolrObject $obj

Return values

array The returned document as an array.

◆ supports_group_filtering()

search_solr\engine::supports_group_filtering ( )

Solr includes group support in the execute_query function.

Return values

bool True

Reimplemented from core_search\engine.

◆ supports_users()

search_solr\engine::supports_users ( )

Solr supports search by user id.

Return values

bool True

Reimplemented from core_search\engine.

◆ to_document()

core_search\engine::to_document	(	\core_search\base	$searcharea,
			$docdata
	)

protectedinherited

Returns a document instance prepared to be rendered.

Parameters

core_search\base	$searcharea
array	$docdata

Return values

core_search

◆ update_schema()

search_solr\engine::update_schema	(	$oldversion,
		$newversion
	)

protected

Requests the search engine to upgrade the schema.

The engine should update the schema if possible/necessary, and should ensure that record_applied_schema_version is called as a result.

If it is not possible to upgrade the schema at the moment, it can do nothing and return; the function will be called again next time search is initialised.

The default implementation just returns, with a DEBUG_DEVELOPER warning.

Parameters

int	$oldversion	Old schema version
int	$newversion	New schema version

Return values

bool|string True if schema is updated successfully, a string if it needs updating manually

Reimplemented from core_search\engine.

Member Data Documentation

◆ $cachedusers

stdClass [] core_search\engine::$cachedusers = array()

staticprotectedinherited

User data required to show their fullnames.

Indexed by userid.

◆ FRAG_SIZE

const search_solr\engine::FRAG_SIZE = 510

Highlighting fragsize.

Slightly larger than output size (500) to allow for ... appending.

The documentation for this class was generated from the following file:

search/engine/solr/classes/engine.php

Public Member Functions

Static Public Member Functions

Public Attributes

Protected Member Functions

Static Protected Member Functions

Protected Attributes

Static Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __construct()

Member Function Documentation

◆ add_document()

◆ add_documents()

◆ add_fields()

◆ add_highlight_content()

◆ add_solr_document()

◆ add_stored_file()

◆ area_index_complete()

◆ area_index_starting()

◆ check_latest_schema()

◆ clear_query_error()

◆ clear_users_cache()

◆ commit()

◆ convert_file_results()

◆ create_user_query()

◆ delete()

◆ delete_by_id()

◆ delete_index_for_context()

◆ delete_index_for_course()

◆ execute_query()

◆ file_indexing_enabled()

◆ file_is_indexable()

◆ get_connection_url()

◆ get_course()

◆ get_curl_object()

◆ get_document_classname()

◆ get_indexed_files()

◆ get_missing_docs()

◆ get_plugin_name()

◆ get_query_error()

◆ get_query_response()

◆ get_query_total_count()

◆ get_response_counts()

◆ get_search_area()

◆ get_search_client()

◆ get_solr_major_version()

◆ get_supported_orders()

◆ get_user()

◆ grouped_files_process_response()

◆ index_complete()

◆ index_starting()

◆ is_installed()

◆ is_server_configured()

◆ is_server_ready()

◆ merge_highlight_field_values()

◆ optimize()

◆ process_document_files()

◆ process_response()

◆ record_applied_schema_version()

◆ replace_underlines()

◆ set_query()

◆ should_skip_schema_check()

◆ skip_schema_check()

◆ standarize_solr_obj()

◆ supports_group_filtering()

◆ supports_users()

◆ to_document()

◆ update_schema()

Member Data Documentation

◆ $cachedusers

◆ FRAG_SIZE