The API appears to be down

[JoeyD473]

The API appears to be down. Got the production general useless error message:

We're sorry, but something went wrong.

If you are the application owner check the logs for more information.

I get it with all the examples and direct links

[surenm]

hey @JoeyD473, the database behind the site was costing us money so had taken it down which is why the site is down. I can may be help you set it up in your local maybe?

[sputnikOS]

@surenm is it possible to set the data up locally? I was hoping to possibly use the API, but am in the same boat as the user that posted the issue.

[JoeyD473]

I would love to set it up locally

[sputnikOS]

@JoeyD473 is this possible? Would love to get my hands on some data if possible.

[surenm]

Yes, it's possible to host the service locally. Let me write a doc for you folks to follow through. Give me until this weekend to get this done!

…

On Sun 4 Feb, 2018, 12:20 AM Stephen Sabsowitz, ***@***.***> wrote: @JoeyD473 <https://github.com/joeyd473> is this possible? Would love to get my hands on some data if possible. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#48 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAXyPnrgM9cJKOxLQrRy1y7T0W9RlZjYks5tRKpwgaJpZM4OZCby> .

[sputnikOS]

@surenm thanks. That would be awesome.

[surenm]

@sabsowitz @JoeyD473 - i am trying to create a docker container out of this. that seems to be the simplest way to bundle all the deps, so that people who use it just need to run the container. Would that be good enough for you?

[sputnikOS]

@surenm that would be great!

[JohnnyDevNull]

i'am also interested in a local database running. Have a babylonjs and angular project for simulation running and will use this api for it.

[JohnnyDevNull]

Are there any news? Would be nice to get another one running! 🙂

How much ressources would your api need?

[sputnikOS]

@JohnnyDevNull @surenm I haven't heard anything further since my last comment. I would be willing to set up a database and API on one of my servers. Hell, I would just be happy with a .tar file of all the raw data.

I originally was trying to get the raw data and set up my own API if the devs would be alright with it. Even just on my local machine.

[JohnnyDevNull]

Help me to find a good and nearly complete and useful data set and i will build a new one with the same calls from scratch within a week. But with php7 and slim rest framework. Very powerful with a ngix server.

[nicholalexander]

Hello all! Sorry, I actually had this thread muted so am just now seeing it. I can work on getting the data files and things to you tomorrow - the data sets are all open from the museum so it shouldn't be too difficult! Will update in the morning. Thanks all for your interest! - sent from my mobile device.

…

On Thu, May 3, 2018, 8:16 PM Philipp John ***@***.***> wrote: Help me to find a good and nearly complete and useful data set and i will build a new one with the same calls from scratch within a week. But with php7 and slim rest framework. Very powerful with a ngix server. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#48 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAM9QCcGXdpKWCBJISCyJxV2_ZgOU9Dyks5tu532gaJpZM4OZCby> .

[JoeyD473]

Sounds good Joey Derrico On Thu, May 3, 2018 at 8:41 PM, nichol alexander <[email protected]> wrote:

…

Hello all! Sorry, I actually had this thread muted so am just now seeing it. I can work on getting the data files and things to you tomorrow - the data sets are all open from the museum so it shouldn't be too difficult! Will update in the morning. Thanks all for your interest! - sent from my mobile device. On Thu, May 3, 2018, 8:16 PM Philipp John ***@***.***> wrote: > Help me to find a good and nearly complete and useful data set and i will > build a new one with the same calls from scratch within a week. But with > php7 and slim rest framework. Very powerful with a ngix server. > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#48 (comment)- 386474263>, > or mute the thread > <https://github.com/notifications/unsubscribe- auth/AAM9QCcGXdpKWCBJISCyJxV2_ZgOU9Dyks5tu532gaJpZM4OZCby> > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#48 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAcE8anqJd_XYK4twKTt9GHTwO8lmVLOks5tu6PGgaJpZM4OZCby> .

[sputnikOS]

@JohnnyDevNull using Angular 5 and Mongo over here. I am really digging Elixir, so I am thinking of using the datasets with Elixir/Phoenix

[sputnikOS]

@nicholalexander that would be greatly appreciated. Thanks

[nicholalexander]

@JohnnyDevNull @sabsowitz @JoeyD473

You can get the data sets here: http://research.amnh.org/users/abbott/dudata/ - the file you want is DUmilkyway.zip and the files are the speck files and the label files. If you look at the lib/tasks in the repo, you'll get a sense of how we parsed the files to get a clean data set, the details of which I don't remember at the moment.

Hope this helps! Otherwise, it would be nice to have the API back up but I'm not sure I have the time to work on it more right now. Let me know if you do something with it with Elixir - would love to see that!

[JohnnyDevNull]

@nicholalexander nice thing, thanks for the data! My goal is definitely to setup a api with a huge amount of stars, planetes and so on data. But as i posted before it's hard to find a nearly actual complete dataset out there to get it public. The best i have found is this: http://vo.ari.uni-heidelberg.de/ppmxl but with nearly no documentation. Thats hard to get into the astronomy stuff.

@sabsowitz i'am also going to use Angular 5 here in combination with BabylonJS or ThreeJS. Haven't decided yet which will be better for my needs. I have setup for both a template project with running example code:

https://github.com/JohnnyDevNull/ng-babylon-template
https://github.com/JohnnyDevNull/ng-three-template

I want to build a stars/space simulation for our solar system and outer stars. Inspired by Googles 100000 Stars project: http://stars.chromeexperiments.com/

[sputnikOS]

@JohnnyDevNull coming from the nightmare that was AngularJS, Angular 5 is fantastic. I have app that uses NASA' APIs and have found it so much easier to rewrite the app in Angular than migrate it over. Also check out some of HackTheUniverse's other repos. There is some cool 3D stuff in there.

[sputnikOS]

@nicholalexander Thanks. This is greatly appreciated. Super excited to get my hands on your data set. I plan on setting up an API, and with your permission, possibly allow others to use it.

Elixir is my favorite new language. If I make anything of substance with your dataset, I will be sure to let you know!

[sputnikOS]

@nicholalexander What program do you use to open the speck files? I'm not talking about parsing, but just viewing the contents.

[nicholalexander]

Hey! You should be able to load it via Partiview / Digital Universe.

https://www.amnh.org/our-research/hayden-planetarium/digital-universe/
http://astroweb.case.edu/lucy/Partiview/howtopart.html

[anzgar01]

Did anyone manage to get a local copy running?
For me the "rake parsers:milkyway:all" seems to fail.
Thanks!

Edit:
Made some progress with missing dependencies. It seems to try to parse the data, but the result is:
Processing stars.speck...
(0.3ms) BEGIN
SQL (0.5ms) INSERT INTO "stars" ("x", "y", "z", "created_at", "updated_at") VALUES ($1, $2, $3, $4, $5) RETURNING "id" [["x", 0.0], ["y", 0.0], ["z", 0.0], ["created_at", "2020-03-16 20:44:20.941685"], ["updated_at", "2020-03-16 20:44:20.941685"]]
(0.6ms) COMMIT

And that is all. Is there a way to redirect the parse command to a local file?

[JoeyD473]

I was not able to, but I ended up busy working on something else then forgot about this. I would also love to hear if anyone got this working and how

…

On Mon, Mar 16, 2020 at 12:05 PM anzgar01 ***@***.***> wrote: Did anyone manage to get a local copy running? For me the "rake parsers:milkyway:all" seems to fail. Thanks! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#48 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADQJ4MO6DU2ZFWPOPMBODLRHZE6JANCNFSM4DTEE3ZA> .

[anzgar01]

I downloaded the file, unpacked and put all the "speck" files into one folder on a local http server (because the parser task it looking for them that way) and changed the path in the parser file.
Now I get errors parsing the files:
Processing stars.speck...
rake aborted!
ActiveRecord::UnknownAttributeError: unknown attribute 'BVcolor' for Star.

the stars.speck file looks like this:

some commented lines

datavar 0 BVcolor
datavar 1 lum
datavar 2 Vabsmag
...

So it fails parsing the file. Anyone got a clue to why this happens?

[anzgar01]

Alright, I found the problem:
obviously the format of the files has changed. For instance, the data file has the "BVcolor" column, the database column is named "colorb_v", which explains the mismatch.
Does anybody by chance still have the original milkyway.tgz file? That would solve the problem I guess.
Edit:
If you download Digital Universe from http://data.openspaceproject.com/release/0.15.1/OpenSpace-0.15.1.zip, there are speck files included in the correct format!
Parsing the stars.speck file actually worked, however the other files did not -.- And the api isn't working yet, but getting closer :-)
Edit:
Api is partially working. Getting all stars as json works, individual queries don't. I am unsure wether I am going to continue to put effort into this, as the data does not really fit my needs.
For anybody who wants to try:
the parser.rake file looks in the wrong place, so first thing you need to do is to make the .speck files available somewhere. I had to add some columns to the database tables as well to have them fit the .speck files, but then the parser job actually works. At least partially, but you can take it from there if you like :-)