We developed a BCS connector framework based search connnector. The full crawl is working perfectly well. All Items get propagated to the search system and are queryable. No issues with the full crawl
The issue: The incremental crawl never updates any item in the search index
Incremental crawling is implemented using the changelog based approach. Incremental crawl runs without errors, but never updates any items in the search index. Also, entries in the Crawl database url table are never updated (LastTouchStart)
I'll try to describe my environment as best I can:
Model File:
LobSystem, Type="Custom" (Implemented the SystemUtility based on StructuredRepositorySystemUtility, NamingContainer implements INamingContainer, CustomLobUri class derives from LobUri).
Single Entity, System.Guid unique id, with the following method stereotypes
SpecificFinder
Finder (not marked RootFinder but tried both)
ChangedIdEnumerator
DeletedIdEnumerator
IDEnumerator (when I remove the IDEnumerator from the model file, crawl completes and error is logged to the crawl database that the IDEnumerator is missing)
On Full Crawl, ChangedIdEnumerator and DeletedIdEnumerator get called with lastRuntime cookie (DateTime, value is 1900/1/1, I return null) Finder is called and returns IEnumerable of complete Entities. These entities all end up in the search index correctly.
On Incremental crawl, ChangedIdEnumerator and DeletedIdEnumerator are called. The correct Change Token is passed to the methods. The methods update the change token and then return an IEnumerable<Guid> (my entity has Guid Primary key). I checked in the debugger, these methods do not raise an exception, update the change Token correctly and return valud Ids.
After the ChangedIdEnumerator and DeletedIdEnumerator get called, nothing happens.
In the crawl database, the crawl urls are marked with UseChangeLog=1 and the anchor element has the encoded change token value.
All the parameters (esp. the change cookie passed from the search system to the ChangedIdEnumerator method) are valid. After the method completes, the SPecificFinder IS NOT CALLED, and the crawl completes.
I get everything working ... paging, passing ref and out parameters to methods ... NOTE: the specificFinder method of the entity also works correctly.
DeletedIdEnumerator also works and removes items from the index. DeletedIdEnumerator has the exact same signature as CHangedIdEnumerator and the same filters applies.
All the code loads and executes without errors or exceptions.
My environment has the most recent CU installed (Win Server 2k8R2, SQL 2k8r2, MSS 2010 SP1 plus June 2012 CU/separate FAST Server).
Again ... DeletedIdEnumerator works, ChangedIdEnumerator does not
How my ChangedIdEnumerator gets called
MyCode::ReadChangedIds
System.RuntimeMethodHandle::_InvokeMethodFast
System.Reflection.RuntimeMethodInfo::Invoke
System.Reflection.RuntimeMethodInfo::Invoke
System.Reflection.MethodBase::Invoke
Microsoft.Office.Server.Search.Connector.BDC.StructuredRepositorySystemUtility`1::ExecuteMethodOnProxy
Microsoft.Office.Server.Search.Connector.BDC.StructuredRepositorySystemUtility`1::ExecuteStatic
Microsoft.SharePoint.BusinessData.Runtime.DataClassRuntime::ExecuteInternalWithAuthNFailureRetry
Microsoft.SharePoint.BusinessData.Runtime.DataClassRuntime::ExecuteInternal
Microsoft.SharePoint.BusinessData.Runtime.EntityRuntime::ExecuteInternal
Microsoft.SharePoint.BusinessData.Runtime.EntityRuntime::ExecuteFiltered
Microsoft.SharePoint.BusinessData.Runtime.EntityRuntime::<EnumerateChangedIdentities>b__1
Microsoft.SharePoint.BusinessData.Runtime.EntityInstanceEnumeratorFactory::CreateEntityInstanceEnumerator
Microsoft.SharePoint.BusinessData.Runtime.EntityRuntime::EnumerateChangedIdentities
Microsoft.SharePoint.BusinessData.MetadataModel.Dynamic.Entity::EnumerateChangedIdentities
Microsoft.Office.Server.Search.Connector.BDC.EntityInstanceAsChangeLinkEnumerator::EnumerateFromEntity
Microsoft.Office.Server.Search.Connector.BDC.EntityInstanceEnumerator`1::MoveNext
Microsoft.Office.Server.Search.Connector.BDC.SynchronizableEntity+<EnumerateChanges>d__0::MoveNext
CHelper::AdvanceEnumerator<Microsoft::Office::Server::Search::Connector::IItemChangeInfo>
After the ChangedIdEnumerator returns, INamingContainer.GetAccessUri(IEntityInstance entityInstance, IEntityInstance parentEntityInstance)
gets called for each id returned.
When I look at the MSSCrawlUrlChanges table in the Search Connector database, I see
- Documents in incremental crawls which were deleted are marked correctly
- Documents found in Full crawls after an index reset are marked correctly
- documents updated in a full crawl which existed in the crawl database before the full crawl started are marked correctly (full crawl, full crawl)
- documents updated in an incremental crawl are not listed at all in this table