Skip to content

Commit

Permalink
Merge pull request #80 from datalad/exportdatacode_example
Browse files Browse the repository at this point in the history
Exportdatacode example
  • Loading branch information
bpoldrack authored Jun 18, 2020
2 parents 1b87408 + fc44669 commit 627be18
Showing 1 changed file with 19 additions and 21 deletions.
40 changes: 19 additions & 21 deletions docs/source/exportdatacode.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

Export version-controlled data to OSF and code to Github
********************************************************

Expand Down Expand Up @@ -37,7 +38,7 @@ copying it will prevent your code from execution.
$ datalad create collab_osf
After having created the dataset we want to populate it with some content (just
like in the Handbook).
like in the Handbook). Importantly we don't want to upload this file on Github, only on OSF - in the real world this could be your data that is too large to upload to Github.

.. code-block:: bash
Expand All @@ -47,51 +48,48 @@ like in the Handbook).
-m "add beginners guide on bash" \
-O books/bash_guide.pdf
And we also want to add a text file, which will be saved on Github_.
And we also want to add a text file, which will be saved on Github_ - in your case this could be the code you are using.

.. code-block:: bash
$ cat << EOT > example.txt
This is just an example file just to show the different ways of saving data
in a DataLad Dataset. EOT
$ mkdir code
$ cd code
$ echo "This is just an example file just to show the different ways of saving data in a DataLad Dataset." > example.txt
$ datalad save --to-git -m "created an example.txt"
We now have a Dataset with one file that can be worked on using Github and one
that should be tracked using `git-annex`.

Setting up Github Remote
^^^^^^^^^^^^^^^^^^^^^^^^
We can set-up a Github Remote with the same name basically using
.. code-block:: bash
$ datalad create-sibling-github REPRONAME --github-login GITHUB_NAME
$ git annex copy code/ --to github
Setting up the OSF Remote
^^^^^^^^^^^^^^^^^^^^^^^^^

To use OSF as a storage, you need to provide either your OSF credentials or an OSF access token.
You can create such a token in your account settings (`Personal access token` and then `Create token`), make sure to create a `full_write` token to be able to create OSF projects and upload data to OSF.
You can create such a token in your account settings (`Personal access token` and then `Create token`), make sure to create a `full_write` token to be able to create OSF projects and upload data to OSF.

.. code-block:: bash
$ export OSF_TOKEN=YOUR_TOKEN_FROM_OSF.IO
We are now going to use datalad to create a sibling dataset on OSF - this will create a new dataset called `OSF_PROJECT_NAME` on the OSF account associated with the OSF token in `$OSF_TOKEN`.
We are now going to use datalad to create a sibling dataset on OSF with name `osf` - this will create a new dataset called `OSF_PROJECT_NAME` on the OSF account associated with the OSF token in `$OSF_TOKEN`.

.. code-block:: bash
$ datalad create-sibling-osf OSF_PROJECT_NAME YOUR_OSF_REMOTE_NAME
$ datalad create-sibling-osf -s osf OSF_PROJECT_NAME
Setting up Github Remote
^^^^^^^^^^^^^^^^^^^^^^^^

And finally we are going to copy all files to this new OSF project.
We can set-up a Github Remote with name `github` and include a publish dependency with OSF - that way, when we publish our dataset to Github, the data files get automatically uploaded to OSF.

.. code-block:: bash
$ git annex copy books/* --to YOUR_OSF_REMOTE_NAME
$ datalad create-sibling-github REPRONAME -s github --github-login GITHUB_NAME --publish-depends osf
$ datalad publish . --to github --transfer-data all
This will publish example.txt in code/ to Github and only add the folder structure and symbolic links for all other file; at the same time it will upload the data to OSF - this way you can let OSF handle your data and Github your code.



.. _OSF: https://www.osf.io/
.. _Github: https://www.github.com/

0 comments on commit 627be18

Please sign in to comment.