Features

• The new metadata type "File" supports indexing of arbitrary publication types. Support is in place for both publication level items (MetadataField.PUBLICATION_TYPE_FILE) and article level items (MetadataField.ARTICLE_TYPE_FILE). Article level file items will be assumed to have a publication level file parent even if not explicitly defined. Item metadata beyond the standard access URL, publisher, and provider may be stored as arbitrary key-value pairs in a MetadataField.FIELD_MD_MAP.

• Content Configuration web service now adds AUs from their TDB definition rather than by AUID, matching the way other subsystems add AUs: Including non-definitional parameters, and choosing the least full repository.

• Deep crawl status information (lastDeepCrawl, lastDeepCrawlResult, lastCompletedDeepCrawl, lastCompletedDeepCrawlDepth) is tracked and reported in the UI, and through the getAuStatus() and queryAus() Web services.

• Debug Panel and AU Status now include a "Validate Files" action which runs the plugin's ContentValidator on all files in the AU, reporting any ValidationFailures thrown.

• In lieu of a MIME-type content validator factory, plugins may specify an au_url_mime_validation_map. ValidationFailures will occur for URLs that match one of the patterns but whose Content-Type does not match the corresponding MIME-type. E.g.,

   <entry>
     <string>au_url_mime_validation_map</string>
     <list>
	<string>/doi/pdf(plus)?/, application/pdf</string>
	<string>/doi/(abs|full)/, text/html</string>
     </list>
   </entry>

• ContentValidators may throw ContentValidationException.LogOnly to record a warning message without causing validation failure.

• The "Files" list from AU Status now includes a PollWeight column.

Bug Fixes

• SubscriptionManager omitted non-definitional parameters when adding subscribed AUs.

• The Link Rewriter rewrote in-page links ("#ref"), breaking them.

• Metadata item type inference reversed BOOKCHAPTER and BOOKVOLUME in some circumstances.

• In queryAus() web service, selecting newContentCrawlUrls field caused a fatal error.

• The LastMetadataIndex field in getAuStatus() and queryAus() web services was not accessible using daemonstatusservice.py.

• Fixed unsafe database resource closings and incorrect comparisons in metadata-handling code.

• Fixed active task removal when metadata indexing for an AU is disabled.