Forum » Web Services

Access total listeners data

 
  • Access total listeners data

    Hi,

    I'd like to know which file I have to read to get the total listener data from an artists (or something like a "profile information" for an artist, with total listeners, total plays scrobbled etc.).
    I didn't find any solution to this problem on http://www.audioscrobbler.net/data/webservices/ ...

    Hails

    Totalitarian Alcoholocaust
  • if you sniff the traffic generated by the last.fm client when playing a new track, you will see it request a php page where all the info displayed in the client will be sent at once, you just have to parse it. I can't remember the url, but in case you don't find it, post again and I'll have a look. I haven't found any documentation about it tough.

    Want to listen to your playlists on last.fm? Have a look at TagBar
    • C26000 said...
    • User
    • 1 Aug 2007, 01:02
    hi trascendentalA, can you help me with the php request url?, or where to search for it, I have been browsing this location:
    svn://svn.audioscrobbler.net/client/Audioscrobbler/tags/1.3.1.1/src
    but I really don't know where to search.

  • hi CS2600,
    E.g. for the Artist Buck 65 you have to make a HTTP POST request as an text/xml content type.

    <?xml version="1.0" encoding="UTF-8"?>
    <methodCall>
    <methodName>artistMetadata</methodName>
    <params>
    <param><value><string>buck 65</string></value></param>
    <param><value><string>en</string></value></param>
    </params>
    </methodCall>

    to http://ws.audioscrobbler.com/1.0/rw/xmlrpc.php to get this info back in an xml file.

    Want to listen to your playlists on last.fm? Have a look at TagBar
    • C26000 said...
    • User
    • 2 Aug 2007, 08:09
    it works!,it works!, thanks ! :DD

    I wrote 3 functions in VB.NET to do this, well I actually I copied 2 of them from here :).

    @Heavyraptor, maybe you can use them as a guide if you can understand some VB.NET.


    Public Function GetXmlResponseFromXML(ByVal url As String, ByVal xml As String, ByVal encoding As System.Text.Encoding) As Xml.XmlDocument
    Dim bytes As Byte() = encoding.GetBytes(xml)
    Return GetXmlResponseFromBinary(url, bytes, "text/xml")
    End Function

    Public Function GetXmlResponseFromBinary(ByVal url As String, ByVal bytes As Byte(), ByVal contentType As String) As Xml.XmlDocument
    Dim request As Net.HttpWebRequest = DirectCast(Net.WebRequest.Create(url), Net.HttpWebRequest)
    request.Method = "POST"
    request.ContentLength = bytes.Length
    request.ContentType = contentType
    Using requestStream As IO.Stream = request.GetRequestStream()
    requestStream.Write(bytes, 0, bytes.Length)
    End Using
    Using response As Net.HttpWebResponse = DirectCast(request.GetResponse(), Net.HttpWebResponse)
    If response.StatusCode <> Net.HttpStatusCode.OK Then
    Dim message As String = [String].Format("POST failed. Received HTTP {0}", response.StatusCode)
    Throw New ApplicationException(message)
    End If
    Dim responseSR As IO.StreamReader = New IO.StreamReader(request.GetResponse.GetResponseStream, System.Text.Encoding.UTF8)
    Dim XmlRta As Xml.XmlDocument = New Xml.XmlDocument
    XmlRta.LoadXml(responseSR.ReadToEnd)
    Return XmlRta
    End Using
    End Function

    Private Function GetArtistNumberOfListeners(ByVal ArtistName As String) As Integer
    Try
    Dim XmlRta As Xml.XmlDocument = GetXmlResponseFromXML("http://ws.audioscrobbler.com/1.0/rw/xmlrpc.php";, _
    "<?xml version=""1.0"" encoding=""UTF-8""?>" & _
    "<methodCall>" & _
    "<methodName>artistMetadata</methodName>" & _
    "<params>" & _
    "<param><value><string>" & ArtistName & "</string></value></param>" & _
    "<param><value><string>en</string></value></param>" & _
    "</params>" & _
    "</methodCall>", _
    System.Text.Encoding.UTF8)
    For Each node As Xml.XmlNode In XmlRta("methodResponse")("params")("param")("value")("struct").ChildNodes
    If node("name").InnerText = "numListeners" Then
    Dim NumListenersStr As String = node("value")("string").InnerText
    Return CInt(Val(NumListenersStr))
    End If
    Next
    Return 0
    Catch ex As Exception
    Return 0
    End Try
    End Function


    and again, thanks a lot Philippe, I really needed this. it is almost 8 seconds faster! (using my slow internet connection) than my previous approach.

  • :) Happy I could help you out.

    As a side note, for anyone wanting to get a service from last.fm (or any other site offering web services) which isn't documented, Wireshark (formerly Ethereal) is a great tool to see what goes through any network interface.

    Want to listen to your playlists on last.fm? Have a look at TagBar
  • Hi,

    thanks for your answers.
    Unfortuneatly I don't know any VB.NET, I program in php. But I think I understand the idea beind it.

    I'll post my result's later if it works :).

    Thanks again.

    Totalitarian Alcoholocaust
  • Ok, somehow it isn't working ...
    Are you really sure that it is needed to send that xml header?

    Totalitarian Alcoholocaust
  • heavyraptor said:
    Are you really sure that it is needed to send that xml header?

    Are you asking about the content type or the actual content? Anyways, they both have to be as mentioned above.

    It could have plenty of causes for it not to be working. If you capture an attempt with Wireshak, it should be clear what the problem is. For this, start a new capture on the interface the traffic will go through -> let your program run -> stop the wireshark capture -> locate a packet from the communication -> right click -> follow tcp stream and post the content here

    Want to listen to your playlists on last.fm? Have a look at TagBar
  • Alright I understand ... but I'd like to try with php.
    Anyway, looks like my php still doesn't work ...

    What's wrong with it?

    function get_tl($artist) {
    $data = '<?xml version="1.0" encoding="UTF-8"?'.'>
    <methodCall>
    <methodName>artistMetadata</methodName>
    <params>
    <param><value><string>' . $artist . '</string></value></param>
    <param><value><string>en</string></value></param>
    </params>
    </methodCall>';

    $host = 'www.last.fm';
    $uri = 'http://www.last.fm/1.0/rw/xmlrpc.php';

    if (($fp = @fsockopen($host,80,$errno,$errstr,30)) !== false) {
    $header = "POST $uri HTTP/1.0\r\n";
    $header .= "Host: $host\r\n";
    $header .= "User-Agent: {$_SERVER['HTTP_USER_AGENT']}\r\n";
    $header .= "Content-type: application/x-www-form-urlencoded\r\n";
    $header .= "Content-length: " . strlen($encdata) . "\r\n";
    $header .= "Connection: close\r\n\r\n";

    fputs($fp,$header);
    fputs($fp,$data);

    $buf = '';
    while (!feof($fp))
    $buf .= fgets($fp,1024);

    fclose($fp);
    }

    return $buf;
    }

    Totalitarian Alcoholocaust
  • I'm no php aficionado at all, sorry. If you post a wireshark dump I'll probably be able to tell why it's not working though.
    You might want to read this thread too, as they are trying to solve the same problem.

    Want to listen to your playlists on last.fm? Have a look at TagBar
  • Alright, thanks anyway, I'll take a look at the other thread :).

    Hails

    Totalitarian Alcoholocaust
  • Don't know too much PHP, but I wrote a quick tool to grap this in perl (Yes, I am still in the dark ages). I used an xml-rpc library (XMLRPC::PurePerl) as the interface is by-the-book xml-rpc. If you can find a standard xml-rpc library for PHP it might make your life a lot simpler.

    E-music Profile: riemann
    • exhuma said...
    • User
    • 28 Nov 2007, 08:59
    Going slightly off topic ;) ...

    I am currently trying to receive the same data. In detail I am interested in the total tracks scrobbled for a specific artist. So this already gives me an idea of where to start. BUT... the total number of tracks is hardly a significant metric, as one important dimension is missin.... TIME!

    If a new band pops up, it will naturally have less tracks scrobbled as bands that have been around for ages. It would be useful to receive a value indicating the time when the first track was registered on AS for that band. This would allow some calculations like "tracks scrobbled per hour" for example. Which would yield a much better indication of popularity.

    In the same context, it is already possible in the user-settings to say that the profile page should only show statistics covering the last <x> months. Why is this not accessible over the Webservices? It would be useful in all the webservices accessing the "Top whatever". Being able to supply a start- and end-date would be even more interesting ;)

    Who would be the right guy to address these questions to?

    I was planning to write a small script generating a simple image showcasing some profile statistics. Right now, the statistics I want to calculate are not possible to calculate with the data that is available through the WebServices.

    • C26000 said...
    • User
    • 28 Nov 2007, 14:54
    exhuma said:
    If a new band pops up, it will naturally have less tracks scrobbled as bands that have been around for ages. It would be useful to receive a value indicating the time when the first track was registered on AS for that band. This would allow some calculations like "tracks scrobbled per hour" for example. Which would yield a much better indication of popularity.


    I think that most current bands were created before last.fm so basically the first track scrobled of them depends of their popularity and last.fm popularity. Anyway you can get a good indicator of current popularity with the sum of the weekly top tracks that are shown in the artists pages. and if you want a 'tracks scrobbled per hour' value you can also get it from there just divide the sum of tracks in 168 hours :).

    exhuma said:
    In the same context, it is already possible in the user-settings to say that the profile page should only show statistics covering the last <x> months. Why is this not accessible over the Webservices? It would be useful in all the webservices accessing the "Top whatever". Being able to supply a start- and end-date would be even more interesting ;)


    You can make it yourself using the weekly data, it's a bit longer to execute and implement but it's currently possible.

    exhuma said:
    I was planning to write a small script generating a simple image showcasing some profile statistics. Right now, the statistics I want to calculate are not possible to calculate with the data that is available through the WebServices.


    If you finalize this small script please share it with us at the stats group

  • heavyraptor said:
    Alright I understand ... but I'd like to try with php.
    Anyway, looks like my php still doesn't work ...

    What's wrong with it?


    It looks like you are connecting to www.last.fm and you should be connecting to ws.audioscrobbler.com.
    Also, the full URL isn't needed in the POST.
    I think your code should look like:

    function get_tl($artist) {
    $data = '<?xml version="1.0" encoding="UTF-8"?'.'>
    <methodCall>
    <methodName>artistMetadata</methodName>
    <params>
    <param><value><string>' . $artist . '</string></value></param>
    <param><value><string>en</string></value></param>
    </params>
    </methodCall>';

    $host = 'ws.audioscrobbler.com';
    $uri = '/1.0/rw/xmlrpc.php';

    if (($fp = @fsockopen($host,80,$errno,$errstr,30)) !== false) {
    $header = "POST $uri HTTP/1.0\r\n";
    $header .= "Host: $host\r\n";
    $header .= "User-Agent: {$_SERVER['HTTP_USER_AGENT']}\r\n";
    $header .= "Content-type: application/x-www-form-urlencoded\r\n";
    $header .= "Content-length: " . strlen($encdata) . "\r\n";
    $header .= "Connection: close\r\n\r\n";

    fputs($fp,$header);
    fputs($fp,$data);

    $buf = '';
    while (!feof($fp))
    $buf .= fgets($fp,1024);

    fclose($fp);
    }

    return $buf;
    }

  • Looks like it doesn't work like this either:
    HTTP/1.0 400 Bad request Content-Type: text/html Content-Length: 53 Server: Perlbal Connection: close
    400 - Bad request
    Can't pipeline to HTTP/1.0


    Anyway, I used www.last.fm, because the page I received told me that this file is temporary moved to last.fm and not ws.audioscrobbler.com ... which is actually weird.

    Hails.

    Totalitarian Alcoholocaust
  • I've just read that that can be caused by a missing Content-length header. Although you have Content-length: in your headers, you have the code

    $header .= "Content-length: " . strlen($encdata) . "\r\n";

    but the variable $encdata does not appear to be defined anywhere in your code, which probably means you are sending the Content-length header as zero or blank.
    Are you missing a bit of code to encode the text in $data and store the result in the $encdata variable? If not, you should probably replace $encdata in that line with $data.

  • Ok, the $encdata was wrong, it should be $data, thanks.
    I changed it now, but it still doesn't work (audioscrobbler returns "XML error: no element found at line 1"). The result looks like that:

    HTTP/1.0 200 OK
    Date: Sun, 02 Dec 2007 14:58:27 GMT
    Server: Apache/1.3.33 (Unix) PHP/4.4.3
    X-Proxy-Fix-Up: headers fixed up
    X-Powered-By: PHP/4.4.3
    Connection: close
    Content-Type: text/xml;charset=utf-8

    <?xml version="1.0" encoding="UTF-8"?>
    <methodResponse>
    <fault>
    <value>
    <struct>
    <member>
    <name>faultCode</name>
    <value><int>103</int></value>
    </member>
    <member>
    <name>faultString</name>
    <value><string>XML error: no element found at line 1</string></value>

    </member>
    </struct>
    </value>
    </fault>
    </methodResponse>

    Totalitarian Alcoholocaust
  • I've just tested the code, and it worked for me if I replaced the line
    $header .= "Content-type: application/x-www-form-urlencoded\r\n";
    with
    $header .= "Content-type: text/xml;charset=utf-8\r\n";
    Try that, and see if it works.

  • Hey man, thanks a lot :). It works for me too.
    For the moment I have no further questions.

    Thanks again! :)

    Totalitarian Alcoholocaust
    • galenjr said...
    • User
    • 30 Dec 2007, 07:35
    Get the xml using php and curl

    $data = '<?xml version="1.0" encoding="UTF-8"?><methodCall><methodName>artistMetadata</methodName><params><param><value><string>' . $artist . '</string></value></param><param><value><string>en</string></value></param></params></methodCall>';
    $curl = curl_init();
    curl_setopt($curl, CURLOPT_URL, "http://ws.audioscrobbler.com/1.0/rw/xmlrpc.php";);
    curl_setopt($curl, CURLOPT_POST, TRUE);
    curl_setopt($curl, CURLOPT_RETURNTRANSFER,TRUE);
    curl_setopt($curl, CURLOPT_HTTPHEADER, array("Content-type: text/xml"));
    curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
    $xml = @simplexml_load_string(curl_exec($curl));


    you can then use simplexml to get whatever data you need

  • curl, yeah! it works.
    thank you, man.

    • Daetlus said...
    • User
    • 1 Feb 2008, 03:49
    This sounds to be pointing towards what I'd like to do. Trying to find a way create an excel sheet of the large number of artists I own along with their total plays scrobbled per artist total, and the number of listeners.

    Sadly the last programming I've done was in Pascal.

    If you release a tool, I'd love to hear about it.

    Time to see if I can do me some scriptkitty work to get something working.

Anonymous users may not post messages. Please log in or create an account to post in the forums.