in this post i will first make some observations about sri lankan blog aggregators (kottu in particular) and then make some speculations about how imo sri lankan blogosphere and more specifically its aggregators will evolve in the future. some of it may be considered criticism but hopefully the administrators concerned may show the same tolerance (which i have appreciated before) they have so far demonstrated, if they read this post at all.
before going further i would like of express my appreciation of the service kottu provided/provides and my recognition of the time and money its administrators have expended on it without probably getting any direct benefit (financially speaking). as such what they prefer to do with kottu is completely up to them. following are mere observations and speculations, they and anybody else may make whatever they want to about them.
some personal observations about current kottu features
1/
while visiting kottu i was surprised see my last post published on saturday (3rd) morning listed as one of the 10 most popular posts ( measured presumably by the number of clicks on it) on monday (5th) morning when it was no longer even on the main kottu main page. even more surprisingly it began to go up the rankings. same was observed about two of my posts last week. this popular posts listing in kottu was added while i was taking a few months break except for few almost token posts, so this was the first time i saw it working. only way i could explain this anomaly (since my logs did not indicate that visitors were coming through kuttu page 2 or lower) was by assuming that kottu is including the clicks made on the popular list itself when compiling the list. to make sure before writing this i clicked on my own post several (about 8) times on monday night. while numbers did not change immediately, in the morning i was able to confirm that increase in number of hits in the popular list roughly (though not exactly) matched my clicks plus the number of other visitors to my blog through kottu that night.
imo this sort of self referencing popularity (increase in popularity because of inclusion in the popular list) is not the correct way to go about this.
2/
secondly any popularity list where the turnover is high (as with blog posts) must have a rolling time period during which this popularity is measured. for instance the number of clicks for last 24 hours, 48 hours, seven days, or whatever (imo higher the turnover shorter the period should be). if it's last seven days, monday's list should count the clicks from same time last monday, tuesday's from last tuesday, etc. this however implies that posts in the list should go down gradually as well as going up gradually (if you think about it you will understand why this should be so). instead posts seem to go up and then disappear suddenly. this may indicate an existence of a time limit on the posts, not on the clicks (old posts are taken off after a time no matter how many clicks they receive during the current period).
again imo this is not the correct methodology. primacy should be given to the measure of popularity (actual clicks) even if they are made on old posts, when measuring popularity. however this wrong method may have been the result of above error (if clicks on the items on the list itself are counted those items will tend to stay there forever unless taken off). so if they want to fix this they should fix both together.
3/
third observation i have to make is about the number of blogs that are not included in the kottu blogroll. if you search the location of blogspot bloggers matching "sri lanka"(lk) you will find that most of the listed bloggers are in fact not in kottu. and this is only blogspot. most of such non inclusions do not in fact have active blogs or have cut and paste jobs, but many do have active genuine blogs. there are some blogs of long standing and of roughly the same quality as most blogs in the kottu blogroll (such as Ŀĭ₣є †øŧдĿĿY Rø€K§ и’ §U€K§, jenna, life and beyond, people defined! to take just four in my reader, if you click through the links in them you will find more and more blogs) that are not on kottu. my reader has about 300 sri lankan blogs while kottu has 200 odd. so unless they have asked to be excluded (which i doubt everyone did) kuttu is in a way failing in its purpose.
of course as i said the kottu administrators may have others things to do than go in search of blogs and may prefer the bloggers to come to them requesting inclusion. while understandable this may mean increasing marginalization of kottu itself from the sri lankan blogosphere. see below for some speculations as to what is required.
4/
as i have observed before average kottu blogger tend not to represent real sri lanka. this is inevitable since most early bloggers and the administrators themselves come from what i would call the colombo based english speaking pseudo elite. "pseudo" because their "class" have lost or are losing positions of privilege in sri lankan political and commercial spheres and will lose their position at the top of social hierarchy once the new people who came in to existence ( due to various economic and political changes since 1948, 1956 and esp. 1978 ) learn not to ape them. conscious and unconscious expression of insecurity created by this loss of power is one of the factors that make some of the present bloggers interesting to read, at least imo.
interesting as they may be, to retain their vitality and relevance sri lankan blog aggregators will have to include the increasing numbers of other bloggers from other social classes, as economic growth ( which is after all is going at a very healthy 7.5% ) and spread of internet connectivity introduce ever larger numbers of sri lankans to the blogging. whether current kottu administrators' methods of operation and mindset (in spite of their demonstrated tolerance) can cope with this is an open question. hence the speculations below
6/
all this may have contributed to the relative lessoning of so called kottu effect ( never quite justified if seriously meant as a form of slashdot effect) meaning the increase in traffic once a blog post is in the kottu main page. for instance, at one time about 75 % of my traffic came from kottu. nowadays barely one third (though the absolute number of visitors from there have increased, it is the proportion i am talking about here) comes from kottu even on the days of the post publication. given the number hits in popular list there, other blogs probably get similar numbers. most traffic here comes from search engines and from visitors without any referring page (meaning probably some sort of feed reader, or bookmarks). such visitors tend to be retuning visitors. others come through that most welcome (to me) method, links in others' blog posts (two recent links coming from sri lanka: a land like no other, and pissu balla critics) where they refer to posts in this blog in agreement or disagreement. this indicates that while kottu remains popular it has not kept up with the increase in sri lankan web surfers. its own stats confirm that its traffic has plateaued during last year or so.
5/
as for other sri lankan aggregators, zsiri is a kottu clone with further exclusions, not a real recipe for future success as i will speculate below. achcharu do show more promise in that it seems to be in a constant state of being upgraded with new functionality like tags etc. however they all seem to be minor improvement not the more radical functionality required imo. and it doesn't help that achcharu at present seem to freeze the browser for few seconds while being loaded. perhaps the only site that does this to my firefox, at present.
i think that is about it when it comes to personal observations about kottu, except for the minor one about the current look. i liked the very unique (and very sri lankan) old look; current one imo is very generic. but then who am i to comment about looks, given the look of this blog.
speculations about the future
any future and inclusive blog aggregator will have to handle thousands of blogs and hundreds of posts a day. that means something like the current main page in kottu will be pretty useless since readers will not want to read or even skim through all the posts or even their headings to find what they want.
instead such a blog aggregator should be able to display a custom made page with blogs that are of interest to each particular user. users should have the ability to customize and select the blogs they want to monitor. but this functionality is already available in feed readers and in such places as my yahoo. what would make the sl blog aggregator unique will be its ability to select and suggest new sri lanka related blogs and posts from the hundreds/thousands published that may be of interest to the particular user along with that user's usual blog roll. once it has that ability it will also be able to offer more precise sri lanka related blog search capability than the ones offered by internetwide blog search engines.
all this will require creation of algorithms that can recognize, categorize, and rank, sri lankan blogs and their posts according to subject and preferences of users in the same way as google ranks search results and amazon suggest books to buyers. these algorithms, like the google's, will evaluate users' actions, linkings, tags, etc to do these things. other methods used in social networking sites may also be made use of here.
manual selection and ranking by administrators (except perhaps as an editor's picks section) should be avoided since it would be impracticable and more error prone than one based on users' preferences and algorithms. after all internet has time and time again shown that flexibly structured, broader based, users generated, methods of organizing and presenting information have triumphed over hierarchically structured, tightly controlled, methods of organizing and presenting information. world wide web's flexible hyperlinks won over gopherspace's menus, (in early nineties when i started to use internet people used gopher with its search engines like veronica as much as www. now it is almost extinct), machine indexed search engines (and later ones that ranked results using algorithms based on user links like google) won over human selected search engines like early yahoo directory. only people who have a predilection for control of information and not the benefit of the user will prefer a human selected method over user and algorithm selected method ( no wonder that same people prefer censorship)
such methods can also be extended to posts made in non text media like pics, videos, pod casts etc.
to summarize, a future sri lankan blog aggregator will offer, customized ways to monitor sri lanka related blog posts, interesting blog suggestions, and better search functions, to users, so that they can derive the greatest benefit with least inconvenience from the expected increase in sri lankan blogs
such an aggregator will cost a great deal more to maintain than current kottu, and probably will have to be run on a more business like basis. whether we have reached the point where such an aggregator can earn enough money cover its costs is an open question, that can only be answered by trying. imo there is no doubt that such a point will be reached eventually. so the question is whether anybody is willing to try.
--
update
someone (or two) has been repeatedly clicking on the kottu links for this (17 times altogether) and previous post (12 times) today. probably partially as a result, previous post has reached the top of the kottu popular list with ( with 133 hits). this post has at this time 77 hits in kottu. i am bemused as to what the motivation can be. whatever the reason i would really appreciate if they would leave off doing that and post a comment here giving the reasons.
these took place from (1)14:45:42 to 14:47:10 from an ip address in brisbane australia and (2) from17:23:17 to 17:28:18 from an ip address in uk
update 2
since then further 30 clicks were made to this post from kottu pop. list by the (2) above. i really hope they will stop. - 22.40 02/08/07
--
before going further i would like of express my appreciation of the service kottu provided/provides and my recognition of the time and money its administrators have expended on it without probably getting any direct benefit (financially speaking). as such what they prefer to do with kottu is completely up to them. following are mere observations and speculations, they and anybody else may make whatever they want to about them.
some personal observations about current kottu features
1/
while visiting kottu i was surprised see my last post published on saturday (3rd) morning listed as one of the 10 most popular posts ( measured presumably by the number of clicks on it) on monday (5th) morning when it was no longer even on the main kottu main page. even more surprisingly it began to go up the rankings. same was observed about two of my posts last week. this popular posts listing in kottu was added while i was taking a few months break except for few almost token posts, so this was the first time i saw it working. only way i could explain this anomaly (since my logs did not indicate that visitors were coming through kuttu page 2 or lower) was by assuming that kottu is including the clicks made on the popular list itself when compiling the list. to make sure before writing this i clicked on my own post several (about 8) times on monday night. while numbers did not change immediately, in the morning i was able to confirm that increase in number of hits in the popular list roughly (though not exactly) matched my clicks plus the number of other visitors to my blog through kottu that night.
imo this sort of self referencing popularity (increase in popularity because of inclusion in the popular list) is not the correct way to go about this.
2/
secondly any popularity list where the turnover is high (as with blog posts) must have a rolling time period during which this popularity is measured. for instance the number of clicks for last 24 hours, 48 hours, seven days, or whatever (imo higher the turnover shorter the period should be). if it's last seven days, monday's list should count the clicks from same time last monday, tuesday's from last tuesday, etc. this however implies that posts in the list should go down gradually as well as going up gradually (if you think about it you will understand why this should be so). instead posts seem to go up and then disappear suddenly. this may indicate an existence of a time limit on the posts, not on the clicks (old posts are taken off after a time no matter how many clicks they receive during the current period).
again imo this is not the correct methodology. primacy should be given to the measure of popularity (actual clicks) even if they are made on old posts, when measuring popularity. however this wrong method may have been the result of above error (if clicks on the items on the list itself are counted those items will tend to stay there forever unless taken off). so if they want to fix this they should fix both together.
3/
third observation i have to make is about the number of blogs that are not included in the kottu blogroll. if you search the location of blogspot bloggers matching "sri lanka"(lk) you will find that most of the listed bloggers are in fact not in kottu. and this is only blogspot. most of such non inclusions do not in fact have active blogs or have cut and paste jobs, but many do have active genuine blogs. there are some blogs of long standing and of roughly the same quality as most blogs in the kottu blogroll (such as Ŀĭ₣є †øŧдĿĿY Rø€K§ и’ §U€K§, jenna, life and beyond, people defined! to take just four in my reader, if you click through the links in them you will find more and more blogs) that are not on kottu. my reader has about 300 sri lankan blogs while kottu has 200 odd. so unless they have asked to be excluded (which i doubt everyone did) kuttu is in a way failing in its purpose.
of course as i said the kottu administrators may have others things to do than go in search of blogs and may prefer the bloggers to come to them requesting inclusion. while understandable this may mean increasing marginalization of kottu itself from the sri lankan blogosphere. see below for some speculations as to what is required.
4/
as i have observed before average kottu blogger tend not to represent real sri lanka. this is inevitable since most early bloggers and the administrators themselves come from what i would call the colombo based english speaking pseudo elite. "pseudo" because their "class" have lost or are losing positions of privilege in sri lankan political and commercial spheres and will lose their position at the top of social hierarchy once the new people who came in to existence ( due to various economic and political changes since 1948, 1956 and esp. 1978 ) learn not to ape them. conscious and unconscious expression of insecurity created by this loss of power is one of the factors that make some of the present bloggers interesting to read, at least imo.
interesting as they may be, to retain their vitality and relevance sri lankan blog aggregators will have to include the increasing numbers of other bloggers from other social classes, as economic growth ( which is after all is going at a very healthy 7.5% ) and spread of internet connectivity introduce ever larger numbers of sri lankans to the blogging. whether current kottu administrators' methods of operation and mindset (in spite of their demonstrated tolerance) can cope with this is an open question. hence the speculations below
6/
all this may have contributed to the relative lessoning of so called kottu effect ( never quite justified if seriously meant as a form of slashdot effect) meaning the increase in traffic once a blog post is in the kottu main page. for instance, at one time about 75 % of my traffic came from kottu. nowadays barely one third (though the absolute number of visitors from there have increased, it is the proportion i am talking about here) comes from kottu even on the days of the post publication. given the number hits in popular list there, other blogs probably get similar numbers. most traffic here comes from search engines and from visitors without any referring page (meaning probably some sort of feed reader, or bookmarks). such visitors tend to be retuning visitors. others come through that most welcome (to me) method, links in others' blog posts (two recent links coming from sri lanka: a land like no other, and pissu balla critics) where they refer to posts in this blog in agreement or disagreement. this indicates that while kottu remains popular it has not kept up with the increase in sri lankan web surfers. its own stats confirm that its traffic has plateaued during last year or so.
5/
as for other sri lankan aggregators, zsiri is a kottu clone with further exclusions, not a real recipe for future success as i will speculate below. achcharu do show more promise in that it seems to be in a constant state of being upgraded with new functionality like tags etc. however they all seem to be minor improvement not the more radical functionality required imo. and it doesn't help that achcharu at present seem to freeze the browser for few seconds while being loaded. perhaps the only site that does this to my firefox, at present.
i think that is about it when it comes to personal observations about kottu, except for the minor one about the current look. i liked the very unique (and very sri lankan) old look; current one imo is very generic. but then who am i to comment about looks, given the look of this blog.
speculations about the future
any future and inclusive blog aggregator will have to handle thousands of blogs and hundreds of posts a day. that means something like the current main page in kottu will be pretty useless since readers will not want to read or even skim through all the posts or even their headings to find what they want.
instead such a blog aggregator should be able to display a custom made page with blogs that are of interest to each particular user. users should have the ability to customize and select the blogs they want to monitor. but this functionality is already available in feed readers and in such places as my yahoo. what would make the sl blog aggregator unique will be its ability to select and suggest new sri lanka related blogs and posts from the hundreds/thousands published that may be of interest to the particular user along with that user's usual blog roll. once it has that ability it will also be able to offer more precise sri lanka related blog search capability than the ones offered by internetwide blog search engines.
all this will require creation of algorithms that can recognize, categorize, and rank, sri lankan blogs and their posts according to subject and preferences of users in the same way as google ranks search results and amazon suggest books to buyers. these algorithms, like the google's, will evaluate users' actions, linkings, tags, etc to do these things. other methods used in social networking sites may also be made use of here.
manual selection and ranking by administrators (except perhaps as an editor's picks section) should be avoided since it would be impracticable and more error prone than one based on users' preferences and algorithms. after all internet has time and time again shown that flexibly structured, broader based, users generated, methods of organizing and presenting information have triumphed over hierarchically structured, tightly controlled, methods of organizing and presenting information. world wide web's flexible hyperlinks won over gopherspace's menus, (in early nineties when i started to use internet people used gopher with its search engines like veronica as much as www. now it is almost extinct), machine indexed search engines (and later ones that ranked results using algorithms based on user links like google) won over human selected search engines like early yahoo directory. only people who have a predilection for control of information and not the benefit of the user will prefer a human selected method over user and algorithm selected method ( no wonder that same people prefer censorship)
such methods can also be extended to posts made in non text media like pics, videos, pod casts etc.
to summarize, a future sri lankan blog aggregator will offer, customized ways to monitor sri lanka related blog posts, interesting blog suggestions, and better search functions, to users, so that they can derive the greatest benefit with least inconvenience from the expected increase in sri lankan blogs
such an aggregator will cost a great deal more to maintain than current kottu, and probably will have to be run on a more business like basis. whether we have reached the point where such an aggregator can earn enough money cover its costs is an open question, that can only be answered by trying. imo there is no doubt that such a point will be reached eventually. so the question is whether anybody is willing to try.
--
update
someone (or two) has been repeatedly clicking on the kottu links for this (17 times altogether) and previous post (12 times) today. probably partially as a result, previous post has reached the top of the kottu popular list with ( with 133 hits). this post has at this time 77 hits in kottu. i am bemused as to what the motivation can be. whatever the reason i would really appreciate if they would leave off doing that and post a comment here giving the reasons.
these took place from (1)14:45:42 to 14:47:10 from an ip address in brisbane australia and (2) from17:23:17 to 17:28:18 from an ip address in uk
update 2
since then further 30 clicks were made to this post from kottu pop. list by the (2) above. i really hope they will stop. - 22.40 02/08/07
--
( i have edited out the ip addresses which i published earlier as per a request from someone who thinks her pc was used for this by someone who had the use of it)
i will publish the ip addresses and any other details of future repeated clicking from kottu.
3/
203.51.205.183, CPE-203-51-205-183.qld.bigpond.net.au. , Telstrainternet3, Sydney, Australia clicked 30 times from 18:56:58 to 19:17:22 on 02/09/07 to the next post.
3/
203.51.205.183, CPE-203-51-205-183.qld.bigpond.net.au. , Telstrainternet3, Sydney, Australia clicked 30 times from 18:56:58 to 19:17:22 on 02/09/07 to the next post.
10 comments:
I agree with you with you on the popular posts section. Any post that gets on there immediately gets a hit boost.
After I got about halfway through your post I sort of zoned out. You should have split this up for those of us with ADD. I'll try and finish it later.
Re: Ach freezing the browser. Has this happened recently? If you just checked yesterday, there were persistent server problems :) Yesterday, my browser froze too - because httpd was being DOSed (99% CPU) on the server. That is not a normal occurence. However, if this is your experience on a regular basis, I'd actually like to hear a bit more about it - a mail with some more information would be much appreciated. I obsessed about performance and pageload for Ach2 so your observations would be quite helpful.
I can tell you this much, there is more JavaScript in there than other engines and all of the work you see is done when the page loads - so yes, Achcharu will be slower to launch. Things happen on the client. In Web 2.0 style - slow to launch, but relatively fast to interact :) Or so I tell myself.
For reference:
Kottu currently has a page load of 105kb with 8 entries on the page
Ach has more than three times as many entries visible and a page load of 39kb
[both those numbers only apply if your browser can handle gzip, but most modern browsers can]
Radical functionality? I'd like to hear your ideas on what this constitutes, but I'd refer you to the FAQ first :) Ach has not and *never* will evolve beyond serving a niche. I personally feel that I'm just not interested in the same things as the Kottu administrators (but then, this is just a guess) - so some features have been excluded on purpose. Again, not the time or place to discuss those features perhaps, but feel free to ask. I have reason to believe what I have planned is reasonably radical [for blogs] but *shrug*. It probably isn't.
But thanks for your observations and apologies about a lengthy comment.
Your post is quite interesting. I'm obviously not part of team Kottu so I can't specifically answer any of your points but I do have some observations:
1. I think the top links list is reset often. A future update might include a check for IP perhaps?
2. Your suggestion of having links expire naturally and clicks on the list not be counted (I gather thats what you suggest) is great but it would require some amount of modification on the existing script which (seems to) count all clicks basically.
3. Yes there are many great blogs not included on Kottu. I think this is an issue of scalability and volume - the system Kottu uses isn't designed to syndicate 100s of blogs - if kottu had - say - 400 blogs that might be a bit of a load on the server.
One way to get around this would be to make the script aggregate blogs in batches, but again, this would mean seriously tweaking the way the aggregator works.
In the future (I guess) they may set up a new system which is specifically designed for large scale blog aggregation (like Technorati)?
4. I'm sure Kottu is open to everyone - as you can see even blogs in Sinhala/tamil are included. I don't think they prevent anyone from joining.
Kottu does support categories but nobody seems to use the functionality - I think this is a matter of convenience, also maybe the admins thought it would be an extra hassle as people might switch categories etc.
nsharp: Kottu uses feedwordpress which respects the TTL of a blog feed. This should mean that, configured correctly, not all blogs on your checklist will be pinged at the same time - regardless of the frequency of your cron job. You get rotating checks for free, basically - provided config has been done properly.
sittingnut: erm. I wonder if you'd mind changing the achcharu link to point to one of the domains (achcharu.com or .org) instead of the current location. The present link will be killed off when I move servers and thus will not work for much longer :) thanks.
as you may know indi has written a post dealing with his vision for kottu and with some references to observations in this post. you can read it here
jl:
thanks for the comment. i will try to be more add friendly. :-)
i never try to be 'popular'. i try to be myself, hence the boring long and non capitalized posts.
drac
thanks for the comments. i have changed the link.
as i said i do consider achcharu very promising, and it does have more functionality .
i agree that achcharu is like web 2.0 "slow to launch, but relatively fast to interact". and while the freezing is regular ( and lasts few seconds only, it doesn't crash or anything ) after the site is loaded everything works fine.
keep up the good work and upgrade as you think fit,and as you envision it. everybody has their own visions and it is these visions when acted upon (as you have done) that create the future. so all power to you.
no apologies needed for the length of comment
nsharp
appreciate your comment. thanks
great observations and explanations. i agree with almost all of them.
Please check your gmail, sittingnut? Thanks.
i have deleted my comment at 2/08/2007 7:49 pm, detailing the clicking incident with ip addresses, published just before the first update, as per the request now mentioned at the bottom of in the post.
rest of the original comment was:
"thanks to everyone leaving the comments. i have been rather busy for the last two days so will reply to your comments later. this comment is due to the following activity. i thought i should comment on it as soon as i saw it .
as i will say in an update to the post
someone (or two) has been repeatedly clicking on the kottu links for this (17 times altogether) and previous post (12 times) today. probably partially as a result, previous post has reached the top of the kottu popular list with ( with 133 hits). this post has at this time 77 hits in kottu. i am bemused as to what the motivation can be. whatever the reason i would really appreciate if they would leave off doing that and post a comment here giving the reasons."
I think the trigger happy clicker is Ryan from Queensland (=qld), and his intentions are probably not malevalent. The bugger probably wanted more people to read your post about groundviews censorship.
Needless to say, I'm from Victoria, use a different ISP and have nothing to do with Ryan. Personally, I think Ryan is a spurned Sanjana admirer out for revenge.
Thanks for mentioning my blog Ŀĭ₣є †øŧдĿĿY Rø€K§ и’ §U€K§ in your post...
Post a Comment