Skip to content

Issues: datalad/datalad-crawler

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

36 Open 23 Closed
36 Open 23 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Switch to boto3
#142 opened Jun 6, 2024 by yarikoptic
1
s3: make use of start-after available for list_objects_v2
#132 opened May 8, 2023 by yarikoptic
datalad core testing started to fail
#124 opened Aug 15, 2022 by yarikoptic
2
save versions .json state file reflecting the actual state, not the "target"
#123 opened Aug 11, 2022 by yarikoptic
add option to periodically cause saveing of the result
#122 opened Aug 11, 2022 by yarikoptic
Equivalent of datalad install with datalad crawler question Further information is requested
#114 opened Nov 15, 2021 by N-HEDGER
1
Update Travis config to use new codecov uploader
#110 opened Oct 13, 2021 by jwodder
xnat: automate query to establish a new provider for non-public project
#101 opened Apr 20, 2021 by yarikoptic
3
'CommandError' object has no attribute 'failed'
#91 opened Mar 4, 2021 by yarikoptic
Discontinue use of Repo.add()
#83 opened Oct 13, 2020 by mih
s3: does not detect complete removal of a key
#74 opened Apr 13, 2020 by yarikoptic
s3 crawler -- doesn't handle "gracefully" situation of completely removed keys in a versioned bucket
#73 opened Mar 24, 2020 by yarikoptic
Support (store) ETag if remote (http) resource provides it
#69 opened Feb 11, 2020 by yarikoptic
Add digester to enrich git annex metadata for every key enhancement New feature or request
#65 opened Nov 15, 2019 by yarikoptic
1
NDA crawler fails with error
#56 opened Sep 24, 2019 by loj
8
Crawling multiple web pages
#54 opened Aug 28, 2019 by joeyzhou98
2
--is-pipeline relative import error bug Something isn't working
#52 opened Aug 27, 2019 by joeyzhou98
3
datalad crawl: Changing behaviour between HCP900/1200
#48 opened Jul 16, 2019 by TobiasKadelka
1
Credential lookup failure crashes s3 crawler
#46 opened Jul 11, 2019 by TobiasKadelka
2
Support split archives
#39 opened Mar 4, 2019 by yarikoptic
using the datalad crawler for LORIS
#34 opened Jan 21, 2019 by thomasbeaudry
3
Generic framework for crawling data providers with versions enhancement New feature or request
#22 opened Nov 1, 2018 by yarikoptic
3
"Reproducible" crawled datasets
#21 opened Nov 1, 2018 by yarikoptic
figshare crawler
#20 opened Nov 1, 2018 by yarikoptic
RF: move crawl-init into crawl making crawl accept additional params to be passed into pipeline good first issue Good for newcomers
#19 opened Oct 28, 2018 by mih
@debanjum
ProTip! Type g p on any issue or pull request to go back to the pull request listing page.