.. _plugin-ref: IOPlugin Reference ------------------ :py:class:`IOPlugins ` are used for data import and export for the sources and sinks. The use of is mainly via the URL scheme used to specify input and output ============ ================================================================= scheme :py:class:`IOPlugin ` ============ ================================================================= eucaimdir :ref:`EUCaimDirectory ` eucaimresdir :ref:`EUCaimDirectoryResources ` file :ref:`FileSystem ` strucdir :ref:`StructuredDirectory ` xnat+http :ref:`XNATStorage ` xnat+https :ref:`XNATStorage ` ============ ================================================================= .. _plugin-EUCaimDirectory: EUCaimDirectory ^^^^^^^^^^^^^^^ The EUCAIM plugin is create to handle ``eucaimdir:///`` type or URLs. The URL scheme is rather simple: ``eucaimdir:///path/to/project`` and resembles the FileSystem scheme: (see `wikipedia `_ for details) We do not make use of the ``host`` part and at the moment only support localhost (just leave the host empty) leading to ``eucaimdir:///`` URLs. .. warning:: This plugin ignores the hostname in the URL and does only accept driver letters on Windows in the form ``c:/`` .. _plugin-EUCaimDirectoryResources: EUCaimDirectoryResources ^^^^^^^^^^^^^^^^^^^^^^^^ The EUCAIM plugin is create to handle ``eucaimresdir:///`` type or URLs. This plugin will add sub-folders to reflect multiple types of resources. The URL scheme is rather simple: ``eucaimresdir:///path/to/project`` and resembles the FileSystem scheme: (see `wikipedia `_ for details) We do not make use of the ``host`` part and at the moment only support localhost (just leave the host empty) leading to ``eucaimresdir:///`` URLs. .. warning:: This plugin ignores the hostname in the URL and does only accept driver letters on Windows in the form ``c:/`` .. _plugin-FileSystem: FileSystem ^^^^^^^^^^ The FileSystem plugin is create to handle ``file://`` type or URLs. This is generally not a good practice, as this is not portable over between machines. However, for test purposes it might be useful. The URL scheme is rather simple: ``file://host/path`` (see `wikipedia `_ for details) We do not make use of the ``host`` part and at the moment only support localhost (just leave the host empty) leading to ``file:///`` URLs. .. warning:: This plugin ignores the hostname in the URL and does only accept driver letters on Windows in the form ``c:/`` .. _plugin-StructuredDirectory: StructuredDirectory ^^^^^^^^^^^^^^^^^^^ The Structured plugin is created to handle ``strucdir:///`` type or URLs. The URL scheme is rather simple: ``strucdir:///path/to/project`` and resembles the FileSystem scheme: (see `wikipedia `_ for details) We do not make use of the ``host`` part and at the moment only support localhost (just leave the host empty) leading to ``strucdir:///`` URLs. .. warning:: This plugin ignores the hostname in the URL and does only accept driver letters on Windows in the form ``c:/`` .. _plugin-XNATStorage: XNATStorage ^^^^^^^^^^^ .. warning:: As this IOPlugin is under development, it has not been thoroughly tested. The XNATStorage plugin is an IOPlugin that can download data from and upload data to an XNAT server. It uses its own ``xnat://`` URL scheme. This is a scheme specific for this plugin and though it looks somewhat like the XNAT rest interface, a different type or URL. Data resources can be access directly by a data url:: xnat://xnat.example.com/data/archive/projects/sandbox/subjects/subject001/experiments/experiment001/scans/T1/resources/DICOM xnat://xnat.example.com/data/archive/projects/sandbox/subjects/subject001/experiments/*_BRAIN/scans/T1/resources/DICOM In the second URL you can see a wildcard being used. This is possible at long as it resolves to exactly one item. The ``id`` query element will change the field from the default experiment to subject and the ``label`` query element sets the use of the label as the fastr id (instead of the XNAT id) to ``True`` (the default is ``False``) To disable ``https`` transport and use ``http`` instead the query string can be modified to add ``insecure=true``. This will make the plugin send requests over ``http``:: xnat://xnat.example.com/data/archive/projects/sandbox/subjects/subject001/experiments/*_BRAIN/scans/T1/resources/DICOM?insecure=true For sinks it is import to know where to save the data. Sometimes you want to save data in a new assessor/resource and it needs to be created. To allow the Fastr sink to create an object in XNAT, you have to supply the type as a query parameter:: xnat://xnat.bmia.nl/data/archive/projects/sandbox/subjects/S01/experiments/_BRAIN/assessors/test_assessor/resources/IMAGE/files/image.nii.gz?resource_type=xnat:resourceCatalog&assessor_type=xnat:qcAssessmentData Valid options are: subject_type, experiment_type, assessor_type, scan_type, and resource_type. If you want to do a search where multiple resources are returned, it is possible to use a search url:: xnat://xnat.example.com/search?projects=sandbox&subjects=subject[0-9][0-9][0-9]&experiments=*_BRAIN&scans=T1&resources=DICOM This will return all DICOMs for the T1 scans for experiments that end with _BRAIN that belong to a subjectXXX where XXX is a 3 digit number. By default the ID for the samples will be the experiment XNAT ID (e.g. XNAT_E00123). The wildcards that can be the used are the same UNIX shell-style wildcards as provided by the module :py:mod:`fnmatch`. It is possible to change the id to a different fields id or label. Valid fields are project, subject, experiment, scan, and resource:: xnat://xnat.example.com/search?projects=sandbox&subjects=subject[0-9][0-9][0-9]&experiments=*_BRAIN&scans=T1&resources=DICOM&id=subject&label=true The following variables can be set in the search query: ============= ============== ============================================================================================= variable default usage ============= ============== ============================================================================================= projects ``*`` The project(s) to select, can contain wildcards (see :py:mod:`fnmatch`) subjects ``*`` The subject(s) to select, can contain wildcards (see :py:mod:`fnmatch`) experiments ``*`` The experiment(s) to select, can contain wildcards (see :py:mod:`fnmatch`) scans ``*`` The scan(s) to select, can contain wildcards (see :py:mod:`fnmatch`) resources ``*`` The resource(s) to select, can contain wildcards (see :py:mod:`fnmatch`) id ``experiment`` What field to use a the id, can be: project, subject, experiment, scan, or resource label ``false`` Indicate the XNAT label should be used as fastr id, options ``true`` or ``false`` insecure ``false`` Change the url scheme to be used to http instead of https verify ``true`` (Dis)able the verification of SSL certificates regex ``false`` Change search to use regex :py:func:`re.match` instead of fnmatch for matching overwrite ``false`` Tell XNAT to overwrite existing files if a file with the name is already present ============= ============== ============================================================================================= If you want to download an entire project, you can use an URL in the following format:: xnat://xnat.example.com/projects/project_name This will by default download all the DICOM resources for all subjects, experiments, scans in the project. You can add query parameters just like in the search to control a filter for a subset (or different resource). In practice this works almost like a search with projects='project_name' and resources='DICOM' For storing credentials the ``.netrc`` file can be used. This is a common way to store credentials on UNIX systems. It is required that the file is only accessible by the owner only or a ``NetrcParseError`` will be raised. A netrc file is really easy to create, as its entries look like:: machine xnat.example.com login username password secret123 See the :py:mod:`netrc module ` or the `GNU inet utils website `_ for more information about the ``.netrc`` file. .. note:: On windows the location of the netrc file is assumed to be ``os.path.expanduser('~/_netrc')``. The leading underscore is because windows does not like filename starting with a dot. .. note:: For scan the label will be the scan type (this is initially the same as the series description, but can be updated manually or the XNAT scan type cleanup). .. warning:: labels in XNAT are not guaranteed to be unique, so be careful when using them as the sample ID. This plugin is heavily leaning on `XNATpy ` for the heavy lifting. For background on XNAT, see the `XNAT API DIRECTORY `_ for the REST API of XNAT.