Visualization tool

- gotmagog said...
- User
- 12 Jul 2007, 16:58
Visualization tool

Hi there

I have made a java applet allowing to visualize a graph with vertices users, artists, albums, tags, etc. , from last.fm

it is availalble online here: graph visualization

I am glad somebody told me about the stats group, my applet fits perfectly here. Does anybody know how I can add a link to my applet to the group overview page ?

Have Funn
Gotmagog

[quote][b][user]gotmagog[/user][/b] said: Hi there I have made a java applet allowing to visualize a graph with vertices users, artists, albums, tags, etc. , from last.fm it is availalble online here: [url=www.fmcrawler.de.vu]graph visualization[/url] I am glad somebody told me about the stats group, my applet fits perfectly here. Does anybody know how I can add a link to my applet to the group overview page ? Have Funn Gotmagog[/quote]

Permalink Quote
- C26000 said...
- User
- 12 Jul 2007, 19:45
I really like this :), it's like a more flexible alternative to the music roamer which is one of my favorite stat utilities, I like it because it gives you more control on the graph, you can move any item around freely and you can add any artist individually.

At first I was kind of frustrated because it didn't organized the artists according to their similarity, until I found the cluster button, I think that you should make that clearer it wasn't that obvious, maybe a message somewhere telling you to click 'Cluster' if you want to automatically organize the graph.

I think that It would be cool if you can enter your user name and it create your top 100 or 50 cluster graph automatically similar to this idea

BTW.
How are you calculating the distance between the artists, using the tags or the similar artist list?

I'm going to add it to the overview now

[quote][b][user]C26000[/user][/b] said: I really like this :), it's like a more flexible alternative to the [url=http://www.ccs.neu.edu/home/cash21/bin/MusicRoamer.html]music roamer[/url] which is one of my favorite stat utilities, I like it because it gives you more control on the graph, you can move any item around freely and you can add any artist individually. At first I was kind of frustrated because it didn't organized the artists according to their similarity, until I found the cluster button, I think that you should make that clearer it wasn't that obvious, maybe a message somewhere telling you to click 'Cluster' if you want to automatically organize the graph. I think that It would be cool if you can enter your user name and it create your top 100 or 50 cluster graph automatically similar to [url=http://www.last.fm/group/Stats/forum/46239/_/237493/1#f3913540]this idea[/url] BTW. How are you calculating the distance between the artists, using the tags or the similar artist list? I'm going to add it to the overview now[/quote]

Permalink Quote
- gotmagog said...
- User
- 12 Jul 2007, 21:12
I am glad you like it

I am happy that you like the applet, it is good that somebody hinted me to go to the Stats group.

For now, I am calculating the artist similarities based on the similar artists and I give a bonus if they have the same artists, and the same place in the ranking increases similarity even more.

The user and tag similarity is also calculated thorugh artists.

I may experiment in the future with tag artist similarity and see whether it is better, or I may introduce a more complex similarity measure involving both artist and tag information.

I am finishing my masters degree in Mathematics and Computer Science right now, so making intuitive interfaces is not my strength :-) , but I may make even more interesting data analysis tools in the future.

Have Funn

[quote][b][user]gotmagog[/user][/b] said: I am happy that you like the applet, it is good that somebody hinted me to go to the Stats group. For now, I am calculating the artist similarities based on the similar artists and I give a bonus if they have the same artists, and the same place in the ranking increases similarity even more. The user and tag similarity is also calculated thorugh artists. I may experiment in the future with tag artist similarity and see whether it is better, or I may introduce a more complex similarity measure involving both artist and tag information. I am finishing my masters degree in Mathematics and Computer Science right now, so making intuitive interfaces is not my strength :-) , but I may make even more interesting data analysis tools in the future. Have Funn[/quote]

Permalink Quote
- C26000 said...
- User
- 12 Jul 2007, 23:42
I hadn't noticed that you could add your top 50 artists by adding your self to the graph and then adding all the favorite artists, it was exactly what I wanted :D (top 300 or 100 would be cool too :)), but I have a problem, if I enter my username then show my top 10 artists and finally click 'cluster' the program freezes and doesn't do anything.

[quote][b][user]C26000[/user][/b] said: I hadn't noticed that you could add your top 50 artists by adding your self to the graph and then adding all the favorite artists, it was exactly what I wanted :D (top 300 or 100 would be cool too :)), but I have a problem, if I enter my username then show my top 10 artists and finally click 'cluster' the program freezes and doesn't do anything.[/quote]

Permalink Quote
- C26000 said...
- User
- 13 Jul 2007, 00:09
I linked it,please tell me if that link is ok, or if you want a direct link to the utility.

[quote][b][user]C26000[/user][/b] said: I linked it,please tell me if that link is ok, or if you want a direct link to the utility.[/quote]

Permalink Quote
- gotmagog said...
- User
- 13 Jul 2007, 19:09
reply for C26000

Thanks for adding it, the webpage link is probably better than the applet directly, because the people will read the readme and understand more quickly what they can do.

I did what you did and actually this is not a bug issue, just that I have not explained correctly the cluster button.

The cluster button organizes the type of entities that is selected in the drop down box. So, if you add your name, the drop down is user , and than add your artists, and the drop down is still user, so when you press cluster there is only one user to organize, and nothing changes, just the cursor remains changed, but this is a small harmless bug , the program still functions.

You need to change the drop down to artists, and then the artists will be the type of entity that is clustered. For now, one may cluster either tags, users or artists, in the future I may give the option to organize all three categories simultaneously.

Do you think that now the cluster button makes sense, or what changes can you suggest ? Thank you for saying what bothered you, I want to make the program as intuitive as possible.

Have Funn
Gotmagog

[quote][b][user]gotmagog[/user][/b] said: Thanks for adding it, the webpage link is probably better than the applet directly, because the people will read the readme and understand more quickly what they can do. I did what you did and actually this is not a bug issue, just that I have not explained correctly the [b]cluster[/b] button. The [b]cluster[/b] button organizes the type of entities that is selected in the drop down box. So, if you add your name, the drop down is [b]user[/b] , and than add your artists, and the drop down is still [b]user[/b], so when you press [b]cluster[/b] there is only one user to organize, and nothing changes, just the cursor remains changed, but this is a small harmless bug , the program still functions. You need to change the drop down to [b]artists[/b], and then the artists will be the type of entity that is clustered. For now, one may cluster either tags, users or artists, in the future I may give the option to organize all three categories simultaneously. Do you think that now the [b]cluster [/b]button makes sense, or what changes can you suggest ? Thank you for saying what bothered you, I want to make the program as intuitive as possible. Have Funn Gotmagog[/quote]

Permalink Quote
- C26000 said...
- User
- 13 Jul 2007, 21:59
thanks, now it makes sense, it works without any problem :D

[quote][b][user]C26000[/user][/b] said: thanks, now it makes sense, it works without any problem :D[/quote]

Permalink Quote
- gotmagog said...
- User
- 17 Jul 2007, 16:40
an example of fmcrawler in action

Good that I posted in the stats group, the popularity of my appler grows and now I have 500+ hits on my web page for the last week.

I think that my program may get even more popular if people learn to play with it. For example, i read the discussion about tag clouds and posted how they can be generated with my applet

tag clouds with fmcrawler

[quote][b][user]gotmagog[/user][/b] said: Good that I posted in the stats group, the popularity of my appler grows and now I have 500+ hits on my web page for the last week. I think that my program may get even more popular if people learn to play with it. For example, i read the discussion about tag clouds and posted how they can be generated with my applet [url=http://www.last.fm/forum/21717/_/224695/3#f4303371]tag clouds with fmcrawler[/url][/quote]

Permalink Quote
- C26000 said...
- User
- 17 Jul 2007, 19:54
gotmagog said:
I think that my program may get even more popular if people learn to play with it

yeah that's true , you have so many options and combinations of them that most people will use only the surface of it, what do you think about adding a couple of preset stats, for example one for making your clustered top 50 artists another for your tags with artists and so on, this way people will only need to select a preset stat enter their user name and press enter. I think that people who visit the group for the first time quickly visit all the stats tools, but there are so many of them that they don't spend so much time trying to understand in deep how a certain stat tool works , they want something quick now and maybe if they like it they will come back again later and learn how to use it more in deep, like I'm doing now with your applet :)

[quote][b][user]C26000[/user][/b] said: [quote][b][user]gotmagog[/user][/b] said: I think that my program may get even more popular if people learn to play with it[/quote] yeah that's true , you have so many options and combinations of them that most people will use only the surface of it, what do you think about adding a couple of preset stats, for example one for making your clustered top 50 artists another for your tags with artists and so on, this way people will only need to select a preset stat enter their user name and press enter. I think that people who visit the group for the first time quickly visit all the stats tools, but there are so many of them that they don't spend so much time trying to understand in deep how a certain stat tool works , they want something quick now and maybe if they like it they will come back again later and learn how to use it more in deep, like I'm doing now with your applet :)[/quote]

Permalink Quote
- gotmagog said...
- User
- 9 Aug 2007, 00:21
world map mode

I have added a new world map location mode, displaying a map and artist/user locations on it. It should be easy and intuitive to use, one just presses the "world map" button to go in map mode and then can press any layout, cluster or voronoi button to get out of map mode.

I would appreciate some feedback on this feature, as well as suggestions for artist'S locations parsed badly.

Have Funn
Gotmagog

[quote][b][user]gotmagog[/user][/b] said: I have added a new world map location mode, displaying a map and artist/user locations on it. It should be easy and intuitive to use, one just presses the "world map" button to go in map mode and then can press any layout, cluster or voronoi button to get out of map mode. I would appreciate some feedback on this feature, as well as suggestions for artist'S locations parsed badly. Have Funn Gotmagog[/quote]

Permalink Quote
- C26000 said...
- User
- 9 Aug 2007, 01:53
nice idea, I think that someone asked about something like this before, but at least with my artists there are so many of them that weren't correctly located, so it isn't very informative to me, but it was funny to see 40 below summer from peru :D , mmmmm I think that one of the members is peruvian...anyway , here are the artists that weren't placed correctly in my chart and their incorrect place. I hope it helps

The Advantage : Antartica
40 Below Summer: Perú
Coheed and Cambria: Puerto rico? , haiti? I don't know? an island below cuba
Atreyu: Israel
On the Might of Princes: India
Blindside and Underoath: australia
Thursday, From Autumn to Ashes, Misfits: Europe

another suggestion is to separate a little the artists ,or maybe do it only with US, the US usually get many artists and it gets very crowded. and a small one: the tooltip of the cluster button says 'visualize the location of users'

[quote][b][user]C26000[/user][/b] said: nice idea, I think that someone asked about something like this before, but at least with my artists there are so many of them that weren't correctly located, so it isn't very informative to me, but it was funny to see 40 below summer from peru :D , mmmmm I think that one of the members is peruvian...anyway , here are the artists that weren't placed correctly in my chart and their incorrect place. I hope it helps [artist]The Advantage[/artist] : Antartica [artist]40 Below Summer[/artist]: Perú [artist]Coheed and cambria[/artist]: Puerto rico? , haiti? I don't know? an island below cuba [artist]Atreyu[/artist]: Israel [artist]On the Might of Princes[/artist]: India [artist]Blindside[/artist] and [artist]Underoath[/artist]: australia [artist]Thursday[/artist], [artist]From autumn to ashes[/artist], [artist]Misfits[/artist]: Europe another suggestion is to separate a little the artists ,or maybe do it only with US, the US usually get many artists and it gets very crowded. and a small one: the tooltip of the cluster button says 'visualize the location of users' [/quote]

Permalink Quote
- gotmagog said...
- User
- 9 Aug 2007, 09:41
bugs

-the tooltip bug fixed
-the misfits are OK now
-the advantage fixed
-thursday fixed
-special for USA fans: now locations are shown at exact spot in the USA

such thing will be cool to make for UK as well, which also has quite many music items, but is too small a place

some problems are not so easy to be fixes:
-blindside has many different bands with the same name
--atreyu is set to israel, because they have jericho in their album names, and not a single other word a a clue where they are from

it proved to be more challenging then I thought to make a perfect parser, but it will get better with time.

At least, when one visualizes his friends and other users, it works well, since the location is set explicitly

[quote][b][user]gotmagog[/user][/b] said: -the tooltip bug fixed -the misfits are OK now -the advantage fixed -thursday fixed -special for USA fans: now locations are shown at exact spot in the USA such thing will be cool to make for UK as well, which also has quite many music items, but is too small a place some problems are not so easy to be fixes: -blindside has many different bands with the same name --atreyu is set to israel, because they have jericho in their album names, and not a single other word a a clue where they are from it proved to be more challenging then I thought to make a perfect parser, but it will get better with time. At least, when one visualizes his friends and other users, it works well, since the location is set explicitly[/quote]

Permalink Quote
- C26000 said...
- User
- 9 Aug 2007, 17:49
nice addition the location inside states :) now 36 crazifists is correctly placed in alaska :D, but besides atreyu and blindside there are already some artists misplaced: rata blanca,the advantage,taking back sunday, underoath, on the might of princess, neskimos.

what are you using for the artist countries?

[quote][b][user]C26000[/user][/b] said: nice addition the location inside states :) now 36 crazifists is correctly placed in alaska :D, but besides atreyu and blindside there are already some artists misplaced: rata blanca,the advantage,taking back sunday, underoath, on the might of princess, neskimos. what are you using for the artist countries?[/quote]

Permalink Quote
- gotmagog said...
- User
- 9 Aug 2007, 22:28
the algorithm

I read the artists biographies from lastfm, and then try to parse the correct locations from the text. I use a list of names of countries and cities, generated by the Wordnet ontology, and then the first match inside the biography is returned. This may lead to problems if the name of the artist is a country name, or the name of some album occuring before the real country.

I will try to solve this with time and feedback, but this is a challenging data mining task in itself :-). If lastfm makes an artist country webfeed this will sort itself out.

You may open the java console to see some debug information about what string made the program decide for this country, but this would be really boring for anyone besides the program creator :-)

[quote][b][user]gotmagog[/user][/b] said: I read the artists biographies from lastfm, and then try to parse the correct locations from the text. I use a list of names of countries and cities, generated by the Wordnet ontology, and then the first match inside the biography is returned. This may lead to problems if the name of the artist is a country name, or the name of some album occuring before the real country. I will try to solve this with time and feedback, but this is a challenging data mining task in itself :-). If lastfm makes an artist country webfeed this will sort itself out. You may open the java console to see some debug information about what string made the program decide for this country, but this would be really boring for anyone besides the program creator :-)[/quote]

Permalink Quote
- C26000 said...
- User
- 9 Aug 2007, 23:02
why don't you use wikipedia instead, its faster and easier to parse, take for example the band stratovarious, to find their country just request

http://en.wikipedia.org/wiki/special:export/Stratovarius

and then in the xml you just need to find something like this:

Origin = [[Image:Flag of Finland.svg|25px|Finnish flag]] [[Helsinki]], [[Finland]]

[quote][b][user]C26000[/user][/b] said: why don't you use wikipedia instead, its faster and easier to parse, take for example the band stratovarious, to find their country just request http://en.wikipedia.org/wiki/special:export/Stratovarius and then in the xml you just need to find something like this: Origin = [[Image:Flag of Finland.svg|25px|Finnish flag]] [[Helsinki]], [[Finland]] [/quote]

Permalink Quote
- gotmagog said...
- User
- 10 Aug 2007, 08:31
done

I implemented this right now, seems to be working well.
Thanks for your feedback, it really helps me improve on my program. I alo thought abot wikipedia at first, but I did not expect they have such good information and decided to use other sources. Thanks for reminding me about wikipedia.

If you play some more with the applet and tell me where it makes mistakes now, it will be great.

Have Funn, and thank you one more time for your suggestions.

[quote][b][user]gotmagog[/user][/b] said: I implemented this right now, seems to be working well. Thanks for your feedback, it really helps me improve on my program. I alo thought abot wikipedia at first, but I did not expect they have such good information and decided to use other sources. Thanks for reminding me about wikipedia. If you play some more with the applet and tell me where it makes mistakes now, it will be great. Have Funn, and thank you one more time for your suggestions.[/quote]

Permalink Quote
- C26000 said...
- User
- 10 Aug 2007, 16:35
it has solved a couple of issues, taking back sunday still is in europe :). if you can read VB.NET, sometime ago I wrote a procedure that correctly downloads the artist wiki xml (apparently... :) ), it takes into account the disambiguation and redirection pages and correctly handles pages with (band) or (artist) in their name.

how to use it:

Dim Parameters as string(0 to 1)
Parameters(0) = "Artist"
Parameters(1) = "The Beatles"
DownloadWikipediaArticle("http://en.wikipedia.org/wiki/Special%3aExport/The_Beatles";, "C:\Artist.xml",Parameters)

File:
http://www.sendspace.com/file/a7gova

maybe you can find it useful

[quote][b][user]C26000[/user][/b] said: it has solved a couple of issues, taking back sunday still is in europe :). if you can read VB.NET, sometime ago I wrote a procedure that correctly downloads the artist wiki xml (apparently... :) ), it takes into account the disambiguation and redirection pages and correctly handles pages with (band) or (artist) in their name. how to use it: Dim Parameters as string(0 to 1) Parameters(0) = "Artist" Parameters(1) = "The Beatles" DownloadWikipediaArticle("http://en.wikipedia.org/wiki/Special%3aExport/The_Beatles", "C:\Artist.xml",Parameters) File: http://www.sendspace.com/file/a7gova maybe you can find it useful [/quote]
Edited by C26000 on 10 Aug 2007, 19:26

Permalink Quote
- C26000 said...
- User
- 10 Aug 2007, 16:48
How to make the world map of your top 50 artists.

1. Open:
http://lastfm.atelier-grenzwert.de/

2. In 'Type Press and Enter' select 'User', delete 'type a name ' text, type your username in the box and press enter.

3. Select your picture with left click or drawing a rectangle around it.

4. Right click your picture and select 'favorite artist' from the context menu. Do this step 4 more times in order to add your full top 50 to the graph.

5. Right click your picture and select 'delete'

6. Click 'world map' button in the lower right corner of the page and wait.

7. To navigate the map select 'Transforming' in the lower left corner of the page, and navigate the graph dragging it with the mouse and using the mouse scroll.

Please post here if an artist isn't correctly placed.

[quote][b][user]C26000[/user][/b] said: 1. Open: http://lastfm.atelier-grenzwert.de/ 2. In 'Type Press and Enter' select 'User', delete 'type a name ' text, type your username in the box and press enter. 3. Select your picture with left click or drawing a rectangle around it. 4. Right click your picture and select 'favorite artist' from the context menu. Do this step 4 more times in order to add your full top 50 to the graph. 5. Right click your picture and select 'delete' 6. Click 'world map' button in the lower right corner of the page and wait. 7. To navigate the map select 'Transforming' in the lower left corner of the page, and navigate the graph dragging it with the mouse and using the mouse scroll. Please post here if an artist isn't correctly placed.[/quote]

Permalink Quote
- snyde1 said...
- Subscriber
- 11 Aug 2007, 07:17
Some of these artists are not easily identifiable. (E.g. pictures from one of the other bands of the same name. That's not Toronto! - That raises the question as to what to do when an artist name represents different artists in different geographical locations. The Trooper icon is in Canada (correctly, I think), but there is also a Romanian band of the same name.)

Artist not in right spot and the correct location
Xavier Caféïne = Québec, Canada
Caféïne = Québec, Canada
Poxy = Québec, Canada
Bowling for Soup = Texas, USA
Northern Pikes = Canada
Metric = Ontario, Canada (It could be right, but it looks like it's meant to be on the eastern seaboard of the US)
The Platters = California, USA
Prism = Canada (though the picture is actually of the "other" Prism - location unknown)
Stripper's Union = Canada
Downchild Blues Band = Canada
Gary Numan = England, U.K.
Vulgaires Machins = Québec, Canada
Northey Valenzuela = Northey is from Canada, Valenzuela from the USA. You pick. (Icon is over the Mediterranean.)

I see what's going on with Metric - they're being plunked onto NY city (because they formed there?) and Heart is put into Seattle (same reason?). I'd move both of them to Canada ...

Did I hear that right? Mondegreens - for the misheard word.
Like Odds? Can't get better than Even Odds!

[quote][b][user]snyde1[/user][/b] said: Some of these artists are not easily identifiable. (E.g. pictures from one of the other bands of the same name. That's not [artist]Toronto[/artist]! - That raises the question as to what to do when an artist name represents different artists in different geographical locations. The [artist]trooper[/artist] icon is in Canada (correctly, I think), but there is also a Romanian band of the same name.) Artist not in right spot and the correct location [artist]Xavier Caféïne[/artist] = Québec, Canada [artist]Caféïne[/artist] = Québec, Canada [artist]Poxy[/artist] = Québec, Canada [artist]Bowling for Soup[/artist] = Texas, USA [artist]Northern Pikes[/artist] = Canada [artist]Metric[/artist] = Ontario, Canada (It could be right, but it looks like it's meant to be on the eastern seaboard of the US) [artist]The Platters[/artist] = California, USA [artist]PRism[/artist] = Canada (though the picture is actually of the "other" Prism - location unknown) [artist]Stripper's Union[/artist] = Canada [artist]Downchild Blues Band[/artist] = Canada [artist]gary numan[/artist] = England, U.K. [artist]Vulgaires Machins[/artist] = Québec, Canada [artist]Northey Valenzuela[/artist] = Northey is from Canada, Valenzuela from the USA. You pick. (Icon is over the Mediterranean.) I see what's going on with [artist]Metric[/artist] - they're being plunked onto NY city (because they formed there?) and [artist]Heart[/artist] is put into Seattle (same reason?). I'd move both of them to Canada ...[/quote]

Permalink Quote
- gotmagog said...
- User
- 14 Aug 2007, 10:15
more updates

I have improved the code some more, better disambiguation of state names in USA and other bugs

i have problem with special unicode characters, like Québec, need to learn how to use them in java

Vulgaires Machins has french wiki, so i won't parse them correctly

the bowling for soup is ok now,others also

[quote][b][user]gotmagog[/user][/b] said: I have improved the code some more, better disambiguation of state names in USA and other bugs i have problem with special unicode characters, like Québec, need to learn how to use them in java Vulgaires Machins has french wiki, so i won't parse them correctly the bowling for soup is ok now,others also[/quote]

Permalink Quote
- cantstopdnb said...
- User
- 15 Feb 2009, 17:25
it's off...

shouts! ;)

[quote][b][user]cantstopdnb[/user][/b] said: it's off...[/quote]

Permalink Quote

Anonymous users may not post messages. Please log in or create an account to post in the forums.

Visualization tool

Visualization tool

I am glad you like it

reply for C26000

an example of fmcrawler in action

world map mode

bugs

the algorithm

done

How to make the world map of your top 50 artists.

more updates