Encyclopaedia Metallum: The Metal Archives

Message board

* FAQ    * Search   * Register   * Login 



Reply to topic
Author Message Previous topic | Next topic
doomedlord
Mallcore Kid

Joined: Fri May 13, 2011 6:33 am
Posts: 2
Location: Ireland
PostPosted: Fri May 13, 2011 6:35 am 
 

Hi,

I was wondering if there's an open/public API available for Metal Archives? Searching, retrieving band & discography information, etc.

Thanks,
B.

Top
 Profile  
Evenfiel
Heavy Metal Hunter

Joined: Tue May 27, 2003 9:50 am
Posts: 6145
Location: Brazil
PostPosted: Fri May 13, 2011 7:28 am 
 

So far there is only an Android API:
http://www.androidzoom.com/android_appl ... _fzey.html

Top
 Profile  
doomedlord
Mallcore Kid

Joined: Fri May 13, 2011 6:33 am
Posts: 2
Location: Ireland
PostPosted: Fri May 13, 2011 9:13 am 
 

What I want to know is: how does the Android app access the Metal Archives data? Is it through 1.) connecting directly to the MI.com database, 2.) screen scraping or 3.) querying an API? If it's the latter, I would like to know how I could go about querying it.

Top
 Profile  
Evenfiel
Heavy Metal Hunter

Joined: Tue May 27, 2003 9:50 am
Posts: 6145
Location: Brazil
PostPosted: Fri May 13, 2011 11:24 am 
 

Oh, I see. If you're serious programming an app for MA, you should talk to Morrigan and / or Hellblazer.

Top
 Profile  
Alhadis
Madder Max

Joined: Fri Sep 05, 2008 8:35 am
Posts: 4010
Location: Australia
PostPosted: Fri May 13, 2011 11:39 am 
 

... though I should nonchalantly point out I'm already working on an app for the iPhone.

Just putting that out there. 0=)
_________________
Buy shit.

Top
 Profile  
Merlinthewizard
Mallcore Kid

Joined: Sun Aug 21, 2011 8:00 am
Posts: 10
Location: Germany
PostPosted: Sun Aug 21, 2011 8:02 am 
 

I want an API too!

Top
 Profile  
HellBlazer
Veteran

Joined: Sat Aug 10, 2002 6:48 am
Posts: 3456
Location: Canada
PostPosted: Sun Aug 21, 2011 6:25 pm 
 

There is no API at present time. The Android app just parses the pages to get its data. We will probably have a formal API at some point in the future but I couldn't really say when; the priority is always on improving the site itself.

Top
 Profile  
Merlinthewizard
Mallcore Kid

Joined: Sun Aug 21, 2011 8:00 am
Posts: 10
Location: Germany
PostPosted: Fri Aug 26, 2011 6:06 pm 
 

And exactly there is the problem: if you change the site just a bit, like adding a new color, that won't work anymore.

I would be happy help you to develop an API.

Top
 Profile  
Cursarion
Metalhead

Joined: Mon Nov 21, 2005 6:56 am
Posts: 792
Location: Finland
PostPosted: Fri Aug 26, 2011 8:53 pm 
 

^ Umm... You can't parse information from a page if its colour is changed? Yeeaah.
_________________
Empä mie semmone ou niiku sie luulet

Top
 Profile  
Alhadis
Madder Max

Joined: Fri Sep 05, 2008 8:35 am
Posts: 4010
Location: Australia
PostPosted: Sat Aug 27, 2011 1:07 am 
 

Quote:
And exactly there is the problem: if you change the site just a bit, like adding a new colour, that won't work anymore.

Uhm...
Quote:
I would be happy help you to develop an API.

Erh, mate... not to sound judgemental, but if you write your code based on ad-hoc structure, you're in no position to offer help with coding. o0;
_________________
Buy shit.

Top
 Profile  
Merlinthewizard
Mallcore Kid

Joined: Sun Aug 21, 2011 8:00 am
Posts: 10
Location: Germany
PostPosted: Sat Aug 27, 2011 3:31 am 
 

An example:

There are only some methods to parse a website.
You can handle a website just as text with 'marks'(links and other information) that you wnat to get.
With that you have to search then for the next site.
So... if you search with an regular expression you are able to get smth like: <dd>active</dd> - thats easy
But if the site will change just a bit to make it more dynamic like it is now <dd class="active">Active</dd>
then you can't parse anymore for the bandstatus.

Also you have to get if you search for a band the answer from the server which comes with JSon. Sure again regularexpressions but in that case you must be sure that the link structure hasn't changed... Etc. etc. etc....

So Alhadis tell he how should i wrote code not in ad-hoc structure? Sure i could parse JavaScript too but that would change that much.

Top
 Profile  
Alhadis
Madder Max

Joined: Fri Sep 05, 2008 8:35 am
Posts: 4010
Location: Australia
PostPosted: Sat Aug 27, 2011 4:00 am 
 

Quote:
But if the site will change just a bit to make it more dynamic like it is now <dd class="active">Active</dd>

Then simply equip your RegExp to deal with optional attributes. E.g,
<dd([^\x00]+)?>(\w+)</dd>

In either case, you probably shouldn't be looking so closely at the page's source to extract your data. Simply breaking apart the text content of <div id="band_info"> .. </div> into recognised property values using <dt><dd> would be my approach. :)

If the structure changes in future, than it's your responsibility to remain vigilant and refactor your code. But you're right, we do need a formal API to make the job easier and less hackish.
_________________
Buy shit.

Top
 Profile  
Merlinthewizard
Mallcore Kid

Joined: Sun Aug 21, 2011 8:00 am
Posts: 10
Location: Germany
PostPosted: Sun Aug 28, 2011 8:03 am 
 

@Alhadis thats how i did it with the old Excyclopedia Medtallum Design (That was quite Simple :))

I'll now try to parse a site with a DOM Parser...

Top
 Profile  
Alhadis
Madder Max

Joined: Fri Sep 05, 2008 8:35 am
Posts: 4010
Location: Australia
PostPosted: Sun Aug 28, 2011 8:11 am 
 

And exactly how much has changed, structure-wise?

Swap around a few variables, c'mon, big deal. Image
_________________
Buy shit.

Top
 Profile  
Merlinthewizard
Mallcore Kid

Joined: Sun Aug 21, 2011 8:00 am
Posts: 10
Location: Germany
PostPosted: Mon May 07, 2012 3:13 am 
 

My Java-API: https://code.google.com/p/metalarchives/

Top
 Profile  
GOREG4SM
Mallcore Kid

Joined: Sun Sep 16, 2007 4:24 pm
Posts: 2
PostPosted: Wed Jan 25, 2017 6:34 pm 
 

I realize this topic is five years old now, but it would be great to have an API so you could do things like get a listing of all bands of a genre, sort that list based on average reviews, and then filter out any bands below the 80% mark. Just an idea. I would really like to use the data but at this point I would just have to make a crawler and that would get messy fast.

Top
 Profile  
heavymetalbunny
Mallcore Kid

Joined: Sun Nov 15, 2015 9:50 am
Posts: 1
Location: Belgium
PostPosted: Mon Mar 13, 2017 4:10 am 
 

I'm currently writing a metal music app which is using Metal Archives as one of the data sources. I've built a rudimentary scraper [1], but it would be great to have proper API access to the website. +1

[1] https://github.com/floriandejonckheere/metal_archives

Top
 Profile  
HellBlazer
Veteran

Joined: Sat Aug 10, 2002 6:48 am
Posts: 3456
Location: Canada
PostPosted: Fri Mar 17, 2017 4:18 pm 
 

Someone made this recently as well: http://em.wemakesites.net/

Top
 Profile  
Display posts from previous:  Sort by  
Reply to topic


Who is online

Users browsing this forum: PaganiusI and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

 
Jump to:  

Back to the Encyclopaedia Metallum


Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group