Using the DataONE API on the command line

File 1: dataone-package-upload-example.rst

DataONE Data Package Upload Example

Table of Contents

• Overview
• Choosing a repository
• Gettting an authentication token
• Understanding the create() API call
• Creating system metadata files
• Upload the data files
• Understanding science metadata
• Upload the metadata file
• Understanding resource maps
• Upload the resource map file
• Summary

Overview

This is an example of how to upload content into a repository that implements the DataONE Tier 3 MNStorage API. We focus on a simple yet complete data package example that involves two CSV data files, one EML metadata file, and one OAI-ORE resource map file. This example is based on content in the following published dataset, but has some changes to simplify the example:

Nathan Thorp. 2017. Toolik fertilization experiment 2015 pluck Gas chromatography–mass spectrometry (GCMS) soil measurements. Arctic Data Center. doi:10.18739/A20Q25.

Files in the package:

File	Type	Associated System Metadata File
classes.csv	Data	classes.sysmeta.xml
compounds.csv	Data	compounds.sysmeta.xml
metadata.xml	EML Metadata	metadata.sysmeta.xml
resourcemap.rdf	OAI-ORE Package	resourcemap.sysmeta.xml

Note that while this example uses a shell script to make the HTTP REST calls, this would typically be done programmatically using a software library or application. There are libraries written in Java, Python, R, Matlab, and BASH that implement the DataONE API. The MetacatUI web application also implements this API.

Choosing a repository

We first choose a test Member Node (MN) repository to upload the files. In this example, we will use an NCEAS development server at https://dev.nceas.ucsb.edu. Create a script called upload.sh and make it executable, and add the repository URL as a variable:

#!/bin/bash
mn_url="https://dev.nceas.ucsb.edu/knb/d1/mn";

Gettting an authentication token

1. At the repository page, use the Sign in with ORCID button to log in. You will be redirected to https://orcid.org, where you enter your username and password. Once finished, you will be redirected back to the repository server.
2. Once logged in, choose the My Profile menu item under your account name in the top menu bar. Choose the Settings tab, and in the settings, choose the Authentication Token list item.
3. Use the Copy button to copy your token string to the clipboard.

Paste this token into the top of your upload.sh script. It expires in 18 hours.

#!/bin/bash
mn_url="https://dev.nceas.ucsb.edu/knb/d1/mn";

# token="eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE1MDYyMDc2MjMsInN1Yi ...";
token="<your-token-string-goes-here>";

The token is used later in curl commands sent to the repository. See below.

Understanding the create() API call

The base URL for all programmatic DataONE Member Node API calls for this repository is:

https://dev.nceas.ucsb.edu/knb/d1/mn

Each versioned API endpoint is added to the base URL path. We are most concerned with the MNStorage.create() API call in this example, which is an HTTP POST call to /v2/object:

https://dev.nceas.ucsb.edu/knb/d1/mn/v2/object

The /v2 indicates we are communicating with the repository using version 2 of the DataONE API.

This POST call to /v2/object takes four parameters:

1. session (in the form of the Authentication token as an HTTP Header)
2. pid (the identifier of the object being uploaded)
3. sysmeta (the system metadata XML document describing the object)
4. object (the bytes of the object being uploaded)

This means that to upload our four files, we need to create an associated system metadata file for each. The contents of each file are shown below.

Creating system metadata files

The structure of system metadata XML files follows the DataONE Types schema. We use version 2 of the SystemMetadata Type, which extends version 1.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<d1v2:systemMetadata
    xmlns:d1v1="http://ns.dataone.org/service/types/v1"
    xmlns:d1v2="http://ns.dataone.org/service/types/v2.0">
    <!-- fields go here -->
</d1v2:systemMetadata>

Example system metadata documents are provided below.

The identifier is required, and must be a unique, no whitespace string less than 800 characters. This would typically be a UUID or DOI. Use uuidgen on the command line to create a unique identifier. We use simple identifiers in this example for readability.

The formatId is required, and must come from one of the defined formats in the DataONE Object Formats Registry. Use application/octet-stream if your file type is not in the list, and work with DataONE to expand the list.

The size is required, and must be the exact size of the object in bytes. Use ls -lb <filename> on the command line to get the size in bytes.

The checksum is required, as well as the checksum algorithm. Use shasum -a 1 <filename> to calculate the SHA1 checksum of the file.

The submitter and rightsHolder fields are required, and will typically be the ORCID identifier of the person submitting the content.

The accessPolicy is optional, but is used here to demonstrate how to provide anonymous read access using the public symbolic subject. Without an access policy, the default policy is read, write, and changePermission permissions for only the rightsHolder.

The fileName is optional, but helps tremendously in web-based and other displays of the objects.

Upload the data files

Now that the system metadata documents are assembled, we can upload the data files via the script. Use the curl command to POST the data objects and their system metadata to the repository object service REST endpoint. Add the authentication token as an HTTP Authorization header. The DataONE API transfers content using MIME multipart/form-data as the encoding scheme, which is supported in curl using the -F flag with the @ symbol to define files on disk as parameter values:

#!/bin/bash
mn_url="https://dev.nceas.ucsb.edu/knb/d1/mn";

# token="eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE1MDYyMDc2MjMsInN1Yi ...";
token="<your-token-string-goes-here>";
auth_header="Authorization: Bearer ${token}";

object_endpoint="/v2/object";

# Upload classes.csv
curl -X POST \
    -H "${auth_header}" \
    -F "pid=test.1.1" \
    -F "[email protected]" \
    -F "[email protected]" \
    "${mn_url}${object_endpoint}"

# Upload compounds.csv
curl -X POST \
    -H "${auth_header}" \
    -F "pid=test.2.1" \
    -F "[email protected]" \
    -F "[email protected]" \
    "${mn_url}${object_endpoint}"

Each successful upload of an object will result in the identifier of the new object being returned and an HTTP 200.

Understanding science metadata

Going into detail about the content of the various science metadata standards is beyond the scope of this example. However, DataONE supports multiple community-based, national, and international metadata specifications. In this example, the data files are documented using the Ecological Metadata Language (EML). EML provides elements to describe all of the top-level discovery aspects of the data (who, what, when, where, why), as well as detailed entity and attribute level metadata so that measurements are described fully with definitions, units, etc.

Upload the metadata file

Unsurprisingly, uploading the metadata is the same as uploading the data:

# Upload the science metadata
curl -X POST \
    -H "${auth_header}" \
    -F "pid=test.3.1"\
    -F "[email protected]"\
    -F "[email protected]"\
    "${mn_url}${object_endpoint}"

Understanding resource maps

OAI-ORE resource maps provide a means of packaging objects together into a collection, as well as a way to express virtually any other piece of information about objects because they are constructed as RDF statements. This example uses a plain RDF/XML file since that encoding is the only RDF encoding currently supported by DataONE.

In the below resourcemap.rdf file, each resource being described has an associated dcterms:identifier attribute, and its location is always expressed as a DataONE Coordinating Node resolve URI. By convention, the Aggregation's identifier is the same as the resource map's identifier, but with a #aggregation suffix. This is not required.

The Aggregation is the main concept that is expressed. It defines the collection of data and metadata files using the ore:aggregates predicate. This aggregation is in turn described by a ResourceMap instance in the file, which also provides the modification date and the creator of the resource map.

Each member of the Aggregation is subsequently listed in the file, including the science metadata file and the two data files. The relationships between the science metadata and data are expressed using the cito:documents and cito:isDocumentedBy predicates. These statement bind the metadata and data together in the package.

In more advanced usage, resource maps can aggregate other resource maps to create nested package structures.

Upload the resource map file

Likewise, uploading the resource map is the same as the other files:

# Upload the resource map
curl -X POST \
    -H "${auth_header}" \
    -F "pid=test.4.1"\
    -F "[email protected]"\
    -F "[email protected]"\
    "${mn_url}${object_endpoint}"

Summary

The entire script is as follows:

#!/bin/bash
mn_url="https://dev.nceas.ucsb.edu/knb/d1/mn";

# token="eyJhbGciOiJSUzI1NiJ9.eyJleHAiOjE1MDYyMDc2MjMsInN1Yi ...";
token="<your-token-string-goes-here>";
auth_header="Authorization: Bearer ${token}";

object_endpoint="/v2/object";

# Upload classes.csv
curl -X POST \
    -H "${auth_header}" \
    -F "pid=test.1.1" \
    -F "[email protected]" \
    -F "[email protected]" \
    "${mn_url}${object_endpoint}"

# Upload compounds.csv
curl -X POST \
    -H "${auth_header}" \
    -F "pid=test.2.1" \
    -F "[email protected]" \
    -F "[email protected]" \
    "${mn_url}${object_endpoint}"

# Upload the science metadata
curl -X POST \
    -H "${auth_header}" \
    -F "pid=test.3.1"\
    -F "[email protected]"\
    -F "[email protected]"\
    "${mn_url}${object_endpoint}"

# Upload the resource map
curl -X POST \
    -H "${auth_header}" \
    -F "pid=adc.20.1"\
    -F "[email protected]"\
    -F "[email protected]"\
    "${mn_url}${object_endpoint}"

It defines the repository to upload objects to, an authentication token for the operation, and the service REST endpoint. Using curl, it uploads the four files along with their pre-defined system metadata documents. On success, each new identifier is returned.

File 2: classes.csv

File 3: classes.sysmeta.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<d1v2:systemMetadata 
    xmlns:d1v1="http://ns.dataone.org/service/types/v1" 
    xmlns:d1v2="http://ns.dataone.org/service/types/v2.0">
    <identifier>test.1.1</identifier>
    <formatId>text/csv</formatId>
    <size>3047</size>
    <checksum algorithm="SHA1">c46e888755d55e7f195fae4289cb0907081ac9e1</checksum>
    <submitter>http://orcid.org/0000-0002-8121-2341</submitter>
    <rightsHolder>http://orcid.org/0000-0002-8121-2341</rightsHolder>
    <accessPolicy>
        <allow>
            <subject>public</subject>
            <permission>read</permission>
        </allow>
    </accessPolicy>
    <replicationPolicy replicationAllowed="true" numberReplicas="3"/>
    <fileName>classes.csv</fileName>
</d1v2:systemMetadata>

File 4: compounds.csv

File 5: compounds.sysmeta.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<d1v2:systemMetadata 
    xmlns:d1v1="http://ns.dataone.org/service/types/v1" 
    xmlns:d1v2="http://ns.dataone.org/service/types/v2.0">
    <identifier>test.2.1</identifier>
    <formatId>text/csv</formatId>
    <size>64549</size>
    <checksum algorithm="SHA1">41d1367dc45cee6b15b0af48b35e9bf575cc6845</checksum>
    <submitter>http://orcid.org/0000-0002-8121-2341</submitter>
    <rightsHolder>http://orcid.org/0000-0002-8121-2341</rightsHolder>
    <accessPolicy>
        <allow>
            <subject>public</subject>
            <permission>read</permission>
        </allow>
    </accessPolicy>
    <replicationPolicy replicationAllowed="true" numberReplicas="3"/>
    <fileName>compounds.csv</fileName>
</d1v2:systemMetadata>

File 6: metadata.xml

<?xml version="1.0" encoding="UTF-8"?>
<eml:eml xmlns:eml="eml://ecoinformatics.org/eml-2.1.1" xmlns:stmml="http://www.xml-cml.org/schema/stmml-1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" packageId="doi:10.18739/A20Q25" system="https://arcticdata.io" xsi:schemaLocation="eml://ecoinformatics.org/eml-2.1.1 eml.xsd">
  <dataset>
    <title>Toolik fertilization experiment 2015 pluck Gas chromatography–mass spectrometry (GCMS) soil measurements</title>
    <creator id="7767353682703386">
      <individualName>
        <givenName>Nathan</givenName>
        <surName>Thorp</surName>
      </individualName>
      <organizationName>University of New Hampshire</organizationName>
      <positionName>PhD student</positionName>
      <address>
        <deliveryPoint>UNH Morse Hall 346 8 College Rd</deliveryPoint>
        <city>Durham</city>
        <administrativeArea>New Hampshire</administrativeArea>
        <postalCode>03824</postalCode>
        <country>United States</country>
      </address>
      <phone phonetype="voice">6038622464</phone>
      <electronicMailAddress>[email protected]</electronicMailAddress>
    </creator>
    <metadataProvider id="4910907336484058">
      <individualName>
        <givenName>Nathan</givenName>
        <surName>Thorp</surName>
      </individualName>
      <organizationName>University of New Hampshire</organizationName>
      <positionName>PhD student</positionName>
      <address>
        <deliveryPoint>UNH Morse Hall 346 8 College Rd</deliveryPoint>
        <city>Durham</city>
        <administrativeArea>New Hampshire</administrativeArea>
        <postalCode>03824</postalCode>
        <country>United States</country>
      </address>
      <phone phonetype="voice">6038622464</phone>
      <electronicMailAddress>[email protected]</electronicMailAddress>
    </metadataProvider>
    <associatedParty id="598416036296394">
      <individualName>
        <givenName>Erik</givenName>
        <surName>Hobbie</surName>
      </individualName>
      <organizationName>University of New Hampshire</organizationName>
      <positionName>Research Professor</positionName>
      <address>
        <deliveryPoint>8 College Road</deliveryPoint>
        <city>Durham</city>
        <administrativeArea>New Hampshire</administrativeArea>
        <postalCode>03824</postalCode>
        <country>USA</country>
      </address>
      <phone phonetype="voice">6038623581</phone>
      <electronicMailAddress>[email protected]</electronicMailAddress>
      <role>coPrincipalInvestigator</role>
    </associatedParty>
    <associatedParty id="2630444896800526">
      <individualName>
        <givenName>Laura</givenName>
        <surName>Gough</surName>
      </individualName>
      <organizationName>University of Texas Arlington</organizationName>
      <positionName>Professor</positionName>
      <address>
        <deliveryPoint>701 S. Nedderman Dr (mailbox: 19498)</deliveryPoint>
        <city>Arlington</city>
        <administrativeArea>Texas</administrativeArea>
        <postalCode>76019</postalCode>
        <country>USA</country>
      </address>
      <phone phonetype="voice">817-272-2011</phone>
      <electronicMailAddress>[email protected]</electronicMailAddress>
      <role>coPrincipalInvestigator</role>
    </associatedParty>
    <pubDate>2017-04-26</pubDate>
    <abstract>
      <para>The file contains data from pyrolysis gas chromatography mass spectrometry analysis of carbon compounds in mineral and organic soils taken from a long term fertilization experiment at Toolik Research Station, Alaska. Established in 1981, permafrost plots were fertilized yearly at a rate of 10 grams nitrogen per square meter per year as ammonium nitrate and 5 grams phosphorus per square meter per year as phosphorus pentoxide. Samples were taken from the active layer and include mineral, mixed, and organic soils from the 2015 pluck (harvest). There was a significant (p=0.0387) treatment effect for protein, suggesting a potential shift in microbial populations.</para>
    </abstract>
    <keywordSet>
      <keyword>carbon</keyword>
      <keyword>pyrolysis GCMS</keyword>
      <keyword>arctic</keyword>
      <keyword>soil</keyword>
      <keywordThesaurus>None</keywordThesaurus>
    </keywordSet>
    <intellectualRights>
      <para>
This work is licensed under the Creative Commons Attribution 4.0 International License.
To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
</para>
    </intellectualRights>
    <distribution>
      <offline>
        <mediumName>digital </mediumName>
      </offline>
    </distribution>
    <coverage>
      <geographicCoverage>
        <geographicDescription>Toolik Research Station, Alaska, USA</geographicDescription>
        <boundingCoordinates>
          <westBoundingCoordinate>-149.6128</westBoundingCoordinate>
          <eastBoundingCoordinate>-149.6128</eastBoundingCoordinate>
          <northBoundingCoordinate>68.6294</northBoundingCoordinate>
          <southBoundingCoordinate>68.6294</southBoundingCoordinate>
        </boundingCoordinates>
      </geographicCoverage>
      <temporalCoverage>
        <rangeOfDates>
          <beginDate>
            <calendarDate>1981</calendarDate>
          </beginDate>
          <endDate>
            <calendarDate>2015</calendarDate>
          </endDate>
        </rangeOfDates>
      </temporalCoverage>
      <taxonomicCoverage>
        <taxonomicClassification>
          <taxonRankName>Species</taxonRankName>
          <taxonRankValue>Eriophorum vaginatum</taxonRankValue>
        </taxonomicClassification>
        <taxonomicClassification>
          <taxonRankName>Species</taxonRankName>
          <taxonRankValue>Betula nana</taxonRankValue>
        </taxonomicClassification>
      </taxonomicCoverage>
    </coverage>
    <contact id="1135900638500473">
      <individualName>
        <givenName>Nathan</givenName>
        <surName>Thorp</surName>
      </individualName>
      <organizationName>University of New Hampshire</organizationName>
      <positionName>PhD student</positionName>
      <address>
        <deliveryPoint>UNH Morse Hall 346 8 College Rd</deliveryPoint>
        <city>Durham</city>
        <administrativeArea>New Hampshire</administrativeArea>
        <postalCode>03824</postalCode>
        <country>United States</country>
      </address>
      <phone phonetype="voice">6038622464</phone>
      <electronicMailAddress>[email protected]</electronicMailAddress>
    </contact>
    <methods>
      <methodStep>
        <description>
          <section>
            <title>Fertilized arctic soil collection</title>
            <para>Four replicate 5 x 20 m blocks were fertilized yearly, receiving 10 grams nitrogen per square meter per year as ammonium nitrate and 5 grams phosphorus per square meter per year as phosphorus pentoxide, since 1981. Plots were moist acidic tussock tundra originally dominated by the sedge Eriophorum vaginatum and persisting in the control plots. Nutrient addition plots have become dominated by the deciduous shrub Betula nana, which was originally present as a smaller proportion of the plant community (Shaver and others 2001). Sample names are a composite descriptor where four blocks (B1-B4) each of two treatments; nitrogen and phosphorus addition (NP) or control (CT). The last part of the sample name indicates the horizon. The three soil fractions are indicated in the sample name where O represents the organic horizon, LO (lower organic) is a mixed layer, and M is the mineral horizon. A subset of samples were analyzed using pyrolysis gas chromotography mass spectrometry in the Grandy Lab at the University of New Hampshire. Chromatograms were compared against a library of known compounds and matches were included as percentages of total carbon compounds for each sample. These compounds are also grouped into compound classes in the "Compounds" sheet.</para>
          </section>
        </description>
      </methodStep>
    </methods>
    <project>
      <title>Toolik fertilization experiment 2015 pluck Gas chromatography–mass spectrometry (GCMS) soil measurements</title>
      <personnel>
        <individualName>
          <givenName>Nathan</givenName>
          <surName>Thorp</surName>
        </individualName>
        <role>originator</role>
      </personnel>
      <funding>
        <para>1108074</para>
        <para>1637459</para>
        <para>1026843</para>
        <para>981022</para>
        <para>9211775</para>
      </funding>
    </project>
    <dataTable>
      <entityName>classes.csv</entityName>
      <entityDescription>Data collected frome fertilized arctic soil.</entityDescription>
      <physical id="test.1.1">
        <objectName>classes.csv</objectName>
        <size unit="bytes">3047</size>
        <authentication method="SHA-1">c46e888755d55e7f195fae4289cb0907081ac9e1</authentication>
        <dataFormat>
          <textFormat>
            <numHeaderLines>1</numHeaderLines>
            <recordDelimiter>\n\r</recordDelimiter>
            <attributeOrientation>column</attributeOrientation>
            <simpleDelimited>
              <fieldDelimiter>,</fieldDelimiter>
            </simpleDelimited>
          </textFormat>
        </dataFormat>
        <distribution>
          <online>
            <url function="download">https://cn.dataone.org/cn/v2/resolve/test.1.1</url>
          </online>
        </distribution>
      </physical>
      <attributeList>
        <attribute>
          <attributeName>SampleID</attributeName>
          <attributeDefinition>Sample identifier</attributeDefinition>
          <measurementScale>
            <nominal>
              <nonNumericDomain>
                <textDomain>
                  <definition>Sample identifier</definition>
                </textDomain>
              </nonNumericDomain>
            </nominal>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>ShortName</attributeName>
          <attributeDefinition>Sample name</attributeDefinition>
          <measurementScale>
            <nominal>
              <nonNumericDomain>
                <textDomain>
                  <definition>Sample name</definition>
                </textDomain>
              </nonNumericDomain>
            </nominal>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>Aromatic</attributeName>
          <attributeDefinition>Percentage of sample consisting of aromatic compounds.</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>Lignin</attributeName>
          <attributeDefinition>Percentage of sample consisting of lignin polymers.</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>Lipid</attributeName>
          <attributeDefinition>Percentage of sample consisting of lipids.</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>Polysaccharide</attributeName>
          <attributeDefinition>Percentage of sample consisting of polyssacharides.</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>N_Bearing</attributeName>
          <attributeDefinition>Percentage of sample consisting of nitrogen bearing compounds.</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>Protein</attributeName>
          <attributeDefinition>Percentage of sample consisting of protein.</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>Phenol</attributeName>
          <attributeDefinition>Percentage of sample consisting of phenol.</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>Unkown_Origin</attributeName>
          <attributeDefinition>Percentage of sample of unkown origin.</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>treatment</attributeName>
          <attributeDefinition>What type of treatment the sample received.</attributeDefinition>
          <measurementScale>
            <nominal>
              <nonNumericDomain>
                <enumeratedDomain>
                  <codeDefinition>
                    <code>NA</code>
                    <definition>NA</definition>
                  </codeDefinition>
                </enumeratedDomain>
              </nonNumericDomain>
            </nominal>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>horizon</attributeName>
          <attributeDefinition>Which horizon the sample was taken from.</attributeDefinition>
          <measurementScale>
            <nominal>
              <nonNumericDomain>
                <textDomain>
                  <definition>Which horizon the sample was taken from.</definition>
                </textDomain>
              </nonNumericDomain>
            </nominal>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>block</attributeName>
          <attributeDefinition>Which block the sample was taken from</attributeDefinition>
          <measurementScale>
            <nominal>
              <nonNumericDomain>
                <textDomain>
                  <definition>Which block the sample was taken from</definition>
                </textDomain>
              </nonNumericDomain>
            </nominal>
          </measurementScale>
        </attribute>
      </attributeList>
    </dataTable>
    <dataTable>
      <entityName>compounds.csv</entityName>
      <entityDescription>Information about various compounds.</entityDescription>
      <physical id="test.2.1">
        <objectName>compounds.csv</objectName>
        <size unit="bytes">64549</size>
        <authentication method="SHA-1">41d1367dc45cee6b15b0af48b35e9bf575cc6845</authentication>
        <dataFormat>
          <textFormat>
            <numHeaderLines>1</numHeaderLines>
            <recordDelimiter>\n\r</recordDelimiter>
            <attributeOrientation>column</attributeOrientation>
            <simpleDelimited>
              <fieldDelimiter>,</fieldDelimiter>
            </simpleDelimited>
          </textFormat>
        </dataFormat>
        <distribution>
          <online>
            <url function="download">https://cn.dataone.org/cn/v2/resolve/test.2.1</url>
          </online>
        </distribution>
      </physical>
      <attributeList>
        <attribute>
          <attributeName>Compound</attributeName>
          <attributeDefinition>Compound name.</attributeDefinition>
          <measurementScale>
            <nominal>
              <nonNumericDomain>
                <textDomain>
                  <definition>Compound name</definition>
                </textDomain>
              </nonNumericDomain>
            </nominal>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>Type</attributeName>
          <attributeDefinition>Compound type.</attributeDefinition>
          <measurementScale>
            <nominal>
              <nonNumericDomain>
                <textDomain>
                  <definition>Compound type</definition>
                </textDomain>
              </nonNumericDomain>
            </nominal>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>Source</attributeName>
          <attributeDefinition>Source of compound.</attributeDefinition>
          <measurementScale>
            <nominal>
              <nonNumericDomain>
                <textDomain>
                  <definition>Soure of compound</definition>
                </textDomain>
              </nonNumericDomain>
            </nominal>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B1_CT_LO</attributeName>
          <attributeDefinition>Percentage of compound found in block 1, control section, lower organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B1_CT_M_1</attributeName>
          <attributeDefinition>Percentage of compound found in block 1, control section, mineral horizon, measurement 1</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B1_CT_M_2</attributeName>
          <attributeDefinition>Percentage of compound found in block 1, control section, mineral horizon, measurement 2</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B1_CT_O</attributeName>
          <attributeDefinition>Percentage of compound found in block 1, control section, organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B1_NP_LO</attributeName>
          <attributeDefinition>Percentage of compound found in block 1, nitrogen phosphate added section, lower organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B1_NP_LO_rerun</attributeName>
          <attributeDefinition>Percentage of compound found in block 1, nitrogen phosphate added section, lower organic horizon rerun</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B1_NP_M</attributeName>
          <attributeDefinition>Percentage of compound found in block 1, nitrogen phosphate added section, mineral horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B1_NP_O</attributeName>
          <attributeDefinition>Percentage of compound found in block 1, nitrogen phosphate added section, organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B2_CT_LO</attributeName>
          <attributeDefinition>Percentage of compound found in block 2, control section, lower organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B2_CT_M</attributeName>
          <attributeDefinition>Percentage of compound found in block 2, control section, mineral horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B2_CT_O</attributeName>
          <attributeDefinition>Percentage of compound found in block 2, control section, organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B2_NP_LO_1</attributeName>
          <attributeDefinition>Percentage of compound found in block 2, nitrogen phosphate added section, lower organic horizon, measurement 1</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B2_NP_LO_2</attributeName>
          <attributeDefinition>Percentage of compound found in block 2, nitrogen phosphate added section, lower organic horizon, measurement 2</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B2_NP_M</attributeName>
          <attributeDefinition>Percentage of compound found in block 2, nitrogen phosphate added section, mineral horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B2_NP_O</attributeName>
          <attributeDefinition>Percentage of compound found in block 2, nitrogen phosphate added section, organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B2_NP_O_1</attributeName>
          <attributeDefinition>Percentage of compound found in block 2, nitrogen phosphate added section, organic horizon, second measurement</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B3_CT_LO</attributeName>
          <attributeDefinition>Percentage of compound found in block 3, control section, lower organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B3_CT_M</attributeName>
          <attributeDefinition>Percentage of compound found in block 3, control section, lower organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B3_CT_O</attributeName>
          <attributeDefinition>Percentage of compound found in block 3, control section, organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B3_NP2_LO</attributeName>
          <attributeDefinition>Percentage of compound found in block 3, nitrogen phosphate 2 added, lower organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B3_NP3_LO</attributeName>
          <attributeDefinition>Percentage of compound found in block 3, nitrogen phosphate 3 added, lower organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B3_NP_LO</attributeName>
          <attributeDefinition>Percentage of compound found in block 3, nitrogen phosphate added, lower organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B3_NP_M</attributeName>
          <attributeDefinition>Percentage of compound found in block 3, nitrogen phosphate added, mineral horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B3_NP_O</attributeName>
          <attributeDefinition>Percentage of compound found in block 3, nitrogen phosphate added, organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B4_CT_LO</attributeName>
          <attributeDefinition>Percentage of compound found in block 4, control section, lower organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B4_CT_M</attributeName>
          <attributeDefinition>Percentage of compound found in block 4, control section, mineral horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B4_NP1_LO</attributeName>
          <attributeDefinition>Percentage of compound found in block 4, nitrogen phosphate 1 added, lower organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B4_NP_M</attributeName>
          <attributeDefinition>Percentage of compound found in block 4, nitrogen phosphate added, mineral horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B4_NP_O</attributeName>
          <attributeDefinition>Percentage of compound found in block 4, nitrogen phosphate added, organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
        <attribute>
          <attributeName>B4_NP_LO</attributeName>
          <attributeDefinition>Percentage of compound found in block 4, nitrogen phosphate added, lower organic horizon</attributeDefinition>
          <measurementScale>
            <ratio>
              <unit>
                <standardUnit>dimensionless</standardUnit>
              </unit>
              <numericDomain>
                <numberType>real</numberType>
              </numericDomain>
            </ratio>
          </measurementScale>
        </attribute>
      </attributeList>
    </dataTable>
  </dataset>
</eml:eml>

File 7: metadata.sysmeta.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<d1v2:systemMetadata 
    xmlns:d1v1="http://ns.dataone.org/service/types/v1" 
    xmlns:d1v2="http://ns.dataone.org/service/types/v2.0">
    <identifier>test.3.1</identifier>
    <formatId>eml://ecoinformatics.org/eml-2.1.1</formatId>
    <size>34583</size>
    <checksum algorithm="SHA1">ce2761b17e9466e121ad9f25e930883e4400d36b</checksum>
    <submitter>http://orcid.org/0000-0002-8121-2341</submitter>
    <rightsHolder>http://orcid.org/0000-0002-8121-2341</rightsHolder>
    <accessPolicy>
        <allow>
            <subject>public</subject>
            <permission>read</permission>
        </allow>
    </accessPolicy>
    <replicationPolicy replicationAllowed="true" numberReplicas="3"/>
    <fileName>metadata.xml</fileName>
</d1v2:systemMetadata>

File 8: resourcemap.rdf

<?xml version="1.0" encoding="utf-8"?>
<rdf:RDF
    xmlns:cito="http://purl.org/spar/cito/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
    xmlns:dcterms="http://purl.org/dc/terms/"
    xmlns:foaf="http://xmlns.com/foaf/0.1/"
    xmlns:ore="http://www.openarchives.org/ore/terms/"
    xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
    xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema#">

    <!-- The resource map: resourcemap.rdf -->
    <rdf:Description rdf:about="https://cn.dataone.org/cn/v2/resolve/test.4.1">
        <dcterms:identifier rdf:datatype="http://www.w3.org/2001/XMLSchema#string">test.4.1</dcterms:identifier>
        <rdf:type rdf:resource="http://www.openarchives.org/ore/terms/ResourceMap"/>
        <ore:describes rdf:resource="https://cn.dataone.org/cn/v2/resolve/test.4.1#aggregation"/>
        <dc:creator rdf:nodeID="creator"/>
        <dcterms:modified>2017-09-05T15:14:27-07:00</dcterms:modified>
    </rdf:Description>

    <!-- The resource map creator -->
    <rdf:Description rdf:nodeID="creator">
        <rdf:type rdf:resource="http://purl.org/dc/terms/Agent"/>
        <foaf:name rdf:datatype="http://www.w3.org/2001/XMLSchema#string">DataONE Java Client Library</foaf:name>
    </rdf:Description>

    <!-- The resource map aggregation -->
    <rdf:Description rdf:about="https://cn.dataone.org/cn/v2/resolve/test.4.1#aggregation">
        <dc:title>DataONE Aggregation</dc:title>
        <dcterms:identifier rdf:datatype="http://www.w3.org/2001/XMLSchema#string">test.4.1#aggregation</dcterms:identifier>
        <ore:isDescribedBy rdf:resource="https://cn.dataone.org/cn/v2/resolve/test.4.1#aggregation"/>
        <rdf:type rdf:resource="http://www.openarchives.org/ore/terms/Aggregation"/>
        <ore:aggregates rdf:resource="https://cn.dataone.org/cn/v2/resolve/test.1.1"/>
        <ore:aggregates rdf:resource="https://cn.dataone.org/cn/v2/resolve/test.2.1"/>
        <ore:aggregates rdf:resource="https://cn.dataone.org/cn/v2/resolve/test.3.1"/>
    </rdf:Description>

    <!-- The science metadata: metadata.xml-->
    <rdf:Description rdf:about="https://cn.dataone.org/cn/v2/resolve/test.3.1">
        <dcterms:identifier rdf:datatype="http://www.w3.org/2001/XMLSchema#string">test.3.1</dcterms:identifier>
        <cito:documents rdf:resource="https://cn.dataone.org/cn/v2/resolve/test.1.1"/>
        <cito:documents rdf:resource="https://cn.dataone.org/cn/v2/resolve/test.2.1"/>
        <ore:isAggregatedBy rdf:resource="https://cn.dataone.org/cn/v2/resolve/test.4.1#aggregation"/>
    </rdf:Description>

    <!-- Data file 1: classes.csv-->
    <rdf:Description rdf:about="https://cn.dataone.org/cn/v2/resolve/test.1.1">
        <dcterms:identifier rdf:datatype="http://www.w3.org/2001/XMLSchema#string">test.1.1</dcterms:identifier>
        <cito:isDocumentedBy rdf:resource="https://cn.dataone.org/cn/v2/resolve/test.3.1"/>
        <ore:isAggregatedBy rdf:resource="https://cn.dataone.org/cn/v2/resolve/test.4.1#aggregation"/>
    </rdf:Description>

    <!-- Data file 2: compounds.csv-->
    <rdf:Description rdf:about="https://cn.dataone.org/cn/v2/resolve/test.2.1">
        <dcterms:identifier rdf:datatype="http://www.w3.org/2001/XMLSchema#string">test.2.1</dcterms:identifier>
        <cito:isDocumentedBy rdf:resource="https://cn.dataone.org/cn/v2/resolve/test.3.1"/>
        <ore:isAggregatedBy rdf:resource="https://cn.dataone.org/cn/v2/resolve/test.4.1#aggregation"/>
    </rdf:Description>
</rdf:RDF>

File 9: resourcemap.sysmeta.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<d1v2:systemMetadata 
    xmlns:d1v1="http://ns.dataone.org/service/types/v1" 
    xmlns:d1v2="http://ns.dataone.org/service/types/v2.0">
    <identifier>test.4.1</identifier>
    <formatId>http://www.openarchives.org/ore/terms</formatId>
    <size>3859</size>
    <checksum algorithm="SHA1">9c9713a77b47591ebc576f1224ae0f258d4b6d71</checksum>
    <submitter>http://orcid.org/0000-0002-8121-2341</submitter>
    <rightsHolder>http://orcid.org/0000-0002-8121-2341</rightsHolder>
    <accessPolicy>
        <allow>
            <subject>public</subject>
            <permission>read</permission>
        </allow>
    </accessPolicy>
    <replicationPolicy replicationAllowed="true" numberReplicas="3"/>
    <fileName>resourcemap.rdf</fileName>
</d1v2:systemMetadata>