This page last changed on May 12, 2014 by jive.

The ResourceStore Design outlines an API for stream based access to files in the data directory. Transitioning the codebase to use this API will require a bit of planning.

Planning

GeoServerResourceLoader

  1. Update internals of GeoServerResourceLoader
  2. Add Resource access methods to GeoServerResourceLoader
  3. Remove use of GeoServerResourceLoader search locations
    • WEB_INF/web.xml file access

GeoServerDataDirectory and GeoServerSecurityManager

  1. Update internals of GeoServerDataDirectory
    • Already uses GeoServerResourceLoader
    • Update methods to use Resource internally
  2. Add Resource access methods to GeoServerDataDirectory
  3. Update internals of GeoServerSecurityManager to use Resource
  4. Add Resource access methods to GeoServerSecurityManager

File interaction:

  1. Research
    • Review use org.geoserver.platform.FileWatcher
    • Review use org.geoserver.security.file.FileWatcher
    • Review GWC LockProvider
  2. Port LockProvider and integrate with ResourceStore
    • Bridge from GeoServer LockProvider to GWC LockProvider
  3. Port FileWatcher implementations to use events
    • Update internals of platform FileWatcher so subclasses continue to function unchanged
    • Update internals of security FileWatcher so subclasses continue to function unchanged
  4. Update security to use Resource and Resource events directly rather than FileWatcher

Update GeoServer codebase module by module

    • Direct use of ResourceStore/Resource and Resource.out() for "stream access"
    • Use of Resource.file() for "file access" (i.e. to pass file or URL reference to external library)
    • Replace GeoServerResourceLoader "directory access" with use of Resource

GeoServerResourceLoader

  1. Update internals of GeoServerResourceLoader
    • This "migrates" the codebase to use ResourceStore for lookup
  2. Introduce methods for Resource access
  3. Remove use of GeoServerResourceLoader search locations
    • GeoServerExtensions.file( path ) provided for access to servletContext
    • Fix GeoServerJ2eeRoleService access of WEB_INF/web.xml
  4. Transition checklist (201 references)
    • org.geoserver
    • org.geoserver.catalog
    • org.geoserver.catalog.impl
    • org.geoserver.catalog.rest
    • org.geoserver.catalog.util
    • org.geoserver.config
    • org.geoserver.config.util
    • org.geoserver.csw
    • org.geoserver.csw.store.internal
    • org.geoserver.data.test
    • org.geoserver.ftp
    • org.geoserver.gwc.config
    • org.geoserver.gwc.layer
    • org.geoserver.h2
    • org.geoserver.inspire
    • org.geoserver.jdbcconfig
    • org.geoserver.jdbcconfig.internal
    • org.geoserver.logging
    • org.geoserver.monitor
    • org.geoserver.monitor.auditlog
    • org.geoserver.ows
    • org.geoserver.security
    • org.geoserver.security.impl
    • org.geoserver.security.ldap
    • org.geoserver.spatialite
    • org.geoserver.template
    • org.geoserver.test
    • org.geoserver.wcs
    • org.geoserver.web
    • org.geoserver.web.admin
    • org.geoserver.web.util
    • org.geoserver.web.wicket
    • org.geoserver.wfs
    • org.geoserver.wfs.response
    • org.geoserver.wfs.web
    • org.geoserver.wfs.xml
    • org.geoserver.wfs.xml.v1_1_0
    • org.geoserver.wms
    • org.geoserver.wms.eo
    • org.geoserver.wps
    • org.vfny.geoserver.global
    • org.vfny.geoserver.wms.responses.map.htmlimagemap

web.xml regression

GeoServerJ2eeRoleService makes use of the following:

File webXML = loader.find( "web.xml" );

This is the only example of using search locations to find content outside of the data directory.

Functionality moved to GeoServerExtensions in order to use servlet context:

File webXML = GeoServerExtensions.file( "WEB_INF/web.xml" );

GeoServerDataDirectory

  1. Update internals of GeoServerDataDirectory
  2. Add Resource access methods to GeoServerDataDirectory
  3. Transition to Resource use where possible
  4. Checklist (106 references)
    • org.geoserver.catalog
    • org.geoserver.community.css.web
    • org.geoserver.config
    • org.geoserver.csw.store.simple
    • org.geoserver.data
    • org.geoserver.ftp
    • org.geoserver.gwc
    • org.geoserver.kml.icons
    • org.geoserver.monitor.hib
    • org.geoserver.script
    • org.geoserver.script
    • org.geoserver.security
    • org.geoserver.security.filter
    • org.geoserver.security.impl
    • org.geoserver.security.ldap
    • org.geoserver.template
    • org.geoserver.test
    • org.geoserver.web.admin
    • org.geoserver.wfs.xslt.config
    • org.geoserver.wms
    • org.geoserver.wps
    • org.vfny.geoserver.global

findDataDir / findDataDir regression

The following regressions were found, often due to code taking advantageous of gaps in the implementation of GeoServerDataDirectory.

GWCIntegrationTest:

File h2DefaultStore = dd.findDataFile("gwc/diskquota_page_store_h2");

GetCoverageTest.onSetUp has the following:

File data = getDataDirectory().findDataDir("watertemp");

In these cases I am fixing the test case, but will be more concerned if production code is effected by the dataRoot(false) implementation issues as shown below.

The code is asking to "data/watertemp" but manages to find "watertemp" due to a strange turn of events:

  1. findDataDir("watertemp") is implemented as:
    return dataDir( false, "watertemp" );
    
  2. Where dataDir ends up with:
    return resourceLoader.find( dataRoot(false), "watertemp");
    
  3. Which is where things go wrong, dataRoot(false) fails to find the data directory, resulting in null
    return resourceLoader.find( null, "watertemp");
    
  4. Allowing resourceLoader.find to locate "watertemp" (rather than "data/watertemp").

findStyleFile / findOrCreateStyleFile / findStyleSldFile / findStyleSldFile regression

These methods determine a file location based on a StyleInfo and a filename. A few test cases such as DynamicDimensionsTest made use of a relative path when unpacking class path resources:

testData.addStyle(styleName, "../temperature.sld", getClass(), getCatalog());

This resulted in temperature.sld being unpacked outside the styles directory:

styles/
temperature.sld

And a relative path coming out of StyleInfo:

String path = Paths.path("styles",style.getFilename());
  1. Changed testData add style to ignore relative paths when determining the target filename.
  2. Added Paths.convert( path, filename ) to allow for relative filenames (as long as they stay inside the data directory)

org.vfny.geoserver.global.GeoserverDataDirectory

Action: refactor this one out of existence!

It is mostly used to access the data directory:

File dataDir = GeoserverDataDirectory.getGeoserverDataDirectory();
File storeDir = new File(dataDir, "data/wcs/mosaicfordelete");        

It is also used to resolve file references:

if (path.startsWith("file:")) {
   File fixedPath = GeoserverDataDirectory.findDataFile(path);
   entry.setValue(DataUtilities.fileToURL(fixedPath).toExternalForm());
}
  1. Replace each reference to GeoserverDataDirectory with GeoServerResoruceLoader
  2. Replace GeoserverDataDirectory.getResourceLoader() with GeoServerExtensions.bean(GeoServerResource)
  3. Feedback: Make GeoServerExtensions easier to use
    • Introduced GeoServerExtensionsHelper

GeoServerSecurityManager

  1. Update internals of GeoServerSecurityManager to use Resource
  2. Add Resource access methods to GeoServerSecurityManager
  3. Transition GeoServerSecurityManager direct file access with Resource use where possible
  4. Checklist (106 references)
    • SecurityManager.getSecurityRoot() - 18 items
    • getRoleRoot() - 14 items
    • getPasswordPolicyRoot - 3 items
    • getUserGroupRoot - 13 items
    • getAuthRoot - 3 items
    • getFilterRoot - 5 items
    • getRoot - 3 items
    • getMasterPasswordProviderRoot - 5 items

File Events

  1. Platform FileWatcher
  2. Security FileWatcher
  3. GeoServerTeamplateLoader - looks to be complicated
Document generated by Confluence on May 14, 2014 23:00