IOPlugin Reference¶

IOPlugins are used for data import and export for the sources and sinks.

The use of is mainly via the URL scheme used to specify input and output

scheme	`IOPlugin`
eucaimdir	EUCaimDirectory
eucaimresdir	EUCaimDirectoryResources
file	FileSystem
strucdir	StructuredDirectory
xnat+http	XNATStorage
xnat+https	XNATStorage

EUCaimDirectory¶

The EUCAIM plugin is create to handle eucaimdir:/// type or URLs.

The URL scheme is rather simple: eucaimdir:///path/to/project and resembles the FileSystem scheme: (see wikipedia for details)

We do not make use of the host part and at the moment only support localhost (just leave the host empty) leading to eucaimdir:/// URLs.

Warning

This plugin ignores the hostname in the URL and does only accept driver letters on Windows in the form c:/

EUCaimDirectoryResources¶

The EUCAIM plugin is create to handle eucaimresdir:/// type or URLs. This plugin will add sub-folders to reflect multiple types of resources.

The URL scheme is rather simple: eucaimresdir:///path/to/project and resembles the FileSystem scheme: (see wikipedia for details)

We do not make use of the host part and at the moment only support localhost (just leave the host empty) leading to eucaimresdir:/// URLs.

Warning

This plugin ignores the hostname in the URL and does only accept driver letters on Windows in the form c:/

FileSystem¶

The FileSystem plugin is create to handle file:// type or URLs. This is generally not a good practice, as this is not portable over between machines. However, for test purposes it might be useful.

The URL scheme is rather simple: file://host/path (see wikipedia for details)

We do not make use of the host part and at the moment only support localhost (just leave the host empty) leading to file:/// URLs.

Warning

This plugin ignores the hostname in the URL and does only accept driver letters on Windows in the form c:/

StructuredDirectory¶

The Structured plugin is created to handle strucdir:/// type or URLs.

The URL scheme is rather simple: strucdir:///path/to/project and resembles the FileSystem scheme: (see wikipedia for details)

We do not make use of the host part and at the moment only support localhost (just leave the host empty) leading to strucdir:/// URLs.

Warning

This plugin ignores the hostname in the URL and does only accept driver letters on Windows in the form c:/

XNATStorage¶

Warning

As this IOPlugin is under development, it has not been thoroughly tested.

The XNATStorage plugin is an IOPlugin that can download data from and upload data to an XNAT server. It uses its own xnat:// URL scheme. This is a scheme specific for this plugin and though it looks somewhat like the XNAT rest interface, a different type or URL.

Data resources can be access directly by a data url:

xnat://xnat.example.com/data/archive/projects/sandbox/subjects/subject001/experiments/experiment001/scans/T1/resources/DICOM
xnat://xnat.example.com/data/archive/projects/sandbox/subjects/subject001/experiments/*_BRAIN/scans/T1/resources/DICOM

In the second URL you can see a wildcard being used. This is possible at long as it resolves to exactly one item.

The id query element will change the field from the default experiment to subject and the label query element sets the use of the label as the fastr id (instead of the XNAT id) to True (the default is False)

To disable https transport and use http instead the query string can be modified to add insecure=true. This will make the plugin send requests over http:

xnat://xnat.example.com/data/archive/projects/sandbox/subjects/subject001/experiments/*_BRAIN/scans/T1/resources/DICOM?insecure=true

For sinks it is import to know where to save the data. Sometimes you want to save data in a new assessor/resource and it needs to be created. To allow the Fastr sink to create an object in XNAT, you have to supply the type as a query parameter:

xnat://xnat.bmia.nl/data/archive/projects/sandbox/subjects/S01/experiments/_BRAIN/assessors/test_assessor/resources/IMAGE/files/image.nii.gz?resource_type=xnat:resourceCatalog&assessor_type=xnat:qcAssessmentData

Valid options are: subject_type, experiment_type, assessor_type, scan_type, and resource_type.

If you want to do a search where multiple resources are returned, it is possible to use a search url:

xnat://xnat.example.com/search?projects=sandbox&subjects=subject[0-9][0-9][0-9]&experiments=*_BRAIN&scans=T1&resources=DICOM

This will return all DICOMs for the T1 scans for experiments that end with _BRAIN that belong to a subjectXXX where XXX is a 3 digit number. By default the ID for the samples will be the experiment XNAT ID (e.g. XNAT_E00123). The wildcards that can be the used are the same UNIX shell-style wildcards as provided by the module fnmatch.

It is possible to change the id to a different fields id or label. Valid fields are project, subject, experiment, scan, and resource:

xnat://xnat.example.com/search?projects=sandbox&subjects=subject[0-9][0-9][0-9]&experiments=*_BRAIN&scans=T1&resources=DICOM&id=subject&label=true

The following variables can be set in the search query:

variable	default	usage
projects	`*`	The project(s) to select, can contain wildcards (see `fnmatch`)
subjects	`*`	The subject(s) to select, can contain wildcards (see `fnmatch`)
experiments	`*`	The experiment(s) to select, can contain wildcards (see `fnmatch`)
scans	`*`	The scan(s) to select, can contain wildcards (see `fnmatch`)
resources	`*`	The resource(s) to select, can contain wildcards (see `fnmatch`)
id	`experiment`	What field to use a the id, can be: project, subject, experiment, scan, or resource
label	`false`	Indicate the XNAT label should be used as fastr id, options `true` or `false`
insecure	`false`	Change the url scheme to be used to http instead of https
verify	`true`	(Dis)able the verification of SSL certificates
regex	`false`	Change search to use regex `re.match()` instead of fnmatch for matching
overwrite	`false`	Tell XNAT to overwrite existing files if a file with the name is already present

If you want to download an entire project, you can use an URL in the following format:

xnat://xnat.example.com/projects/project_name

This will by default download all the DICOM resources for all subjects, experiments, scans in the project. You can add query parameters just like in the search to control a filter for a subset (or different resource). In practice this works almost like a search with projects=’project_name’ and resources=’DICOM’

For storing credentials the .netrc file can be used. This is a common way to store credentials on UNIX systems. It is required that the file is only accessible by the owner only or a NetrcParseError will be raised. A netrc file is really easy to create, as its entries look like:

machine xnat.example.com
        login username
        password secret123

See the netrc module or the GNU inet utils website for more information about the .netrc file.

Note

On windows the location of the netrc file is assumed to be os.path.expanduser('~/_netrc'). The leading underscore is because windows does not like filename starting with a dot.

Note

For scan the label will be the scan type (this is initially the same as the series description, but can be updated manually or the XNAT scan type cleanup).

Warning

labels in XNAT are not guaranteed to be unique, so be careful when using them as the sample ID.

This plugin is heavily leaning on XNATpy <https://xnat.readthedocs.io> for the heavy lifting.

For background on XNAT, see the XNAT API DIRECTORY for the REST API of XNAT.