19th Apr 2007

Joomla’s non-optimized search in content.searchbot.php

Hi, All!

Recently I faced a MySQL replication lag time problem. From time to time it increased dramatically (up to 200 sec’s) on new boxes (there are 4 boxes in cluster beyond load balancer) with 4Gb of RAM and really-really nice CPUs. I had to enable slow queries logging to figure out what was causing the problem.
Okay, the slowest query was…. yeah, in search subsystem:

Count: 5 Time=0.80s (4s) Lock=0.00s (0s) Rows=37.2 (186), xxxxx@localhost
SELECT a.title AS title,
a.created AS created,
CONCAT(a.introtext, a.fulltext) AS text,
u.title AS section,
CONCAT( ‘S’, a.id ) AS href,
‘S’ AS browsernav,
‘S’ AS type
FROM jos_content AS a
INNER JOIN jos_categories AS b ON b.id=a.catid
INNER JOIN jos_sections AS u ON u.id = a.sectionid
WHERE ( (LOWER(a.title) LIKE ‘S’ OR LOWER(a.introtext) LIKE ‘S’ OR LOWER(a.fulltext) LIKE ‘S’ OR LOWER(a.metakey) LIKE ‘S’ OR LOWER(a.metadesc) LIKE ‘S’) )
AND a.state = N
AND u.published = N
AND b.published = N
AND a.access <= N
AND b.access <= N
AND u.access <= N
AND ( a.publish_up = 'S' OR a.publish_up <= 'S' )
AND ( a.publish_down = 'S' OR a.publish_down >= ‘S’ )
GROUP BY a.id
ORDER BY a.created DESC
LIMIT N, N

Does anybody have idea why do they use LOWER? MySQL states “By default, MySQL searches are not case sensitive”. So, why add additional load to server?

Anyway, after removing “LOWER” the queries became about 40% faster.

Hope this will help anybody else.

Thanks.

Posted by Posted by Alexander Alfimov under Filed under CMS, Joomla, MySQL Comments 2 Comments »

06th Mar 2007

OpenSEF RC5 SP2 Perfomance Improvement

Hi, all!Recently I’ve rewritten a pretty large site on Joomla (it has about 200 online users all the time, as Joomla backend stats shows, >17000 content items etc.) and noticed that the page generation time started to grow ftom 0.101 secs to 0.5 after the first day, 0.7 - 1 next day…). Not very good, right? First of all I went to the MySQL console at the time of high loads and… voila - there is a `opensef_sef` MySQL query present almost constantly. Guys, can anybody explain me why do they fetch into the memory almost all records in sef.php [Line #173]:

if ($sefSite->id == null) $sefSite->id = -1;
$query = “SELECT * FROM #__opensef_sef WHERE published = ‘1′” .
“\nAND valid = ‘1′” .
“\nAND (direction IS NULL OR direction = ‘’ OR direction = ‘o’)” .
“\nAND (site_id IS NULL ” .
($sefSite->id ? ‘ OR site_id = ‘ . $sefSite->id : ‘’) . ‘)’ .
“\nORDER BY site_id DESC, published DESC, valid DESC, LENGTH(direction) DESC”;
$database->setQuery( $query );
if ($database->loadResult()) {
$opensef_aliases = $database->loadObjectList();
}

A quick fix is to replace the query with this one:

if ($sefSite->id == null) $sefSite->id = -1;
$query = “SELECT * FROM #__opensef_sef WHERE published = ‘1′” .
“\nAND valid = ‘1′” .
“\nAND (direction IS NULL OR direction = ‘’ OR direction = ‘o’)” .
“\nAND (site_id IS NULL ” .
($sefSite->id ? ‘ OR site_id = ‘ . $sefSite->id : ‘’) . ‘)’ .
“\nAND (external=’{$_SERVER[’REQUEST_URI’]}’” .
“\nORDER BY site_id DESC, published DESC, valid DESC, LENGTH(direction) DESC”;
$database->setQuery( $query );
if ($database->loadResult()) {
$opensef_aliases = $database->loadObjectList();
}

After that, find the lookupOutgoing function  in administrator/components/com_sef/sef.class.php and replace it with this one:

function lookupOutgoing( $site_id, $url ) { // static method
global $opensef_aliases, $database;

$returnaliases = array();
/*
if ($site_id == null) $site_id = -1;
if (count($opensef_aliases) > 0) {
foreach($opensef_aliases as $aliases){
if ( $aliases->internal == $url ) {
$returnaliases[] = $aliases;
}
}
}
*/
if (!count($returnaliases))
{
$query = “SELECT * FROM #__opensef_sef WHERE published = ‘1′” .
“\nAND valid = ‘1′” .
“\nAND (direction IS NULL OR direction = ” OR direction = ‘o’)” .
“\nAND internal=’$url’”.
“\nORDER BY site_id DESC, published DESC, valid DESC, LENGTH(direction) DESC limit 1″;

$database->setQuery( $query );
if ($database->loadResult())
{
$aliases = $database->loadObjectList();
foreach($aliases as $k=>$val)
{
$returnaliases[] = $val;
}
}
}

if (is_array( $returnaliases ) && count( $returnaliases )) {
$returnaliases = xclCastObjectList( $returnaliases, ‘JosOpenSEFAlias’ );
return $returnaliases[0];
}
$null = null;
return $null;
}

Hurray! It works as a charm. Page generation time is now 0.07 seconds.

Hope this will help somebody else.

Thanks.

Posted by Posted by Alexander Alfimov under Filed under CMS, Joomla, MySQL, OpenSEF, PHP Comments 1 Comment »

20th Nov 2006

Adsensers or “What’s next?”

Have you ever thought about web without advertisement? Don’t you think it’s possible? May be yes, may be no. But we have to give it a try…

Few years ago I was surprised seeing somebodies project with 3 blocks of google-like ads. “What’s that?”, - I thought, “Where is the content?”. Affiliate links and text ads made up to 80% of the site. It was unusual for 2001-2002 years. But now thousands of web-projects turned from good nice looking source of interesting news etc. into ugly monsters where one should search for the info he came for. And there is a tendence: as more popular is website as more banners it has. Ads on the top, ads on the left, right, bottom and even inside the content! That’s ugly! Pretty project becomes to look like a dirty chicken.

Yes, I understand that is a way to earn some money. But guys, why don’t you try to sell your content itself instead of selling your luck? “May be somebody will click the links and I’ll get a cent… or even dollar!”. People, please try to forget about that. I see more and more adsense-fishing sites without any sort of own content. With such a tendency our children have good chance to get web without something outdated earlier know as “good content”. Even now advertising of advertising is not astonishing.

Why am I writing this? I don’t expect wellknown bloggers and webmasters to remove superfluous. But I hope somebody (even if he’ll be one of a thousand) will read the post and decide to leave only one adsense block on the site instead of putting it everywhere.

Thanks.

Posted by Posted by Alexander Alfimov under Filed under Advertising Comments No Comments »

20th Nov 2006

Joomla vs Drupal

What CMS to choose?

Well, let’s begin from the beginning. What are the most possible candidates? Drupal (http://drupal.org), Joomla (http://joomla.org) and Wordpress (http://wordpress.org – YES! I’m not mistaken, WordPress can be used as CMS). These projects beat anything else I know.

I’d like to begin from the Wordpress. The subject is well-known as blogging engine, but only a few people know it can be used as CMS. Here is what John McCreesh wrote at ONLamp.com:

Could WordPress be used for full-blown websites as well as blogs? I believe the answer is definitely yes. WordPress lets users do an awful lot of things without requiring any knowledge of the underlying technology, such as maintaining content, including images; creating categories for the content; selecting the look-and-feel of the site from a large and growing range of templates; managing multiple users with different access rights; and adding and removing plugins for extra functionality. For users with a working knowledge of web technologies such as HTML, CSS, or PHP, then so much the better — they can dive under the bonnet/hood (depending on which side of the Atlantic you dive from) and work all sorts of magic. The end result is the best of both CMS and blog worlds, with flexible content layout arranged under hierarchical menus such as a CMS, plus all the nice features of blogs such as ease of maintenance, RSS feeds, comments, and permalinks.

As you can see it’s possible to use WordPress as a web-project engine. If you have content-driven project and you do not need e-commerce integration, your business logic is straight-through and simple – WordPress is right for you. A lot of plugins for RSS, polling etc. will help you to build the website quickly and easily.

But what if we need a sort of complex site? Among hundreds of CMSs I distinguish Drupal and Joomla. There is a lot of reasons why these and not other projects, you can check all of them yourself if you don’t trust me. To be honest the only other project that’s worth seeing is Xoops (http://xoops.org). But it’s neither user-friendly nor geniously coded (it is great coded though!) so I’m leaving xoops alone.

Let’s compare those monsters. Monsters in a good meaning. The first thing that caught my eye is Joomla’s admin panel – take a look at http://demo.joomla.org/demo10/administrator/ (alfim/joomlapass). It’s completely user-friendly, isn’t it? Here is what xaneon.com says (Mambo is an older sister of Joomla):

Mambo is certainly “easy on the eyes”: most people react very favorably to seeing the administration interface for the first time. Another aspect where Mambo is definitely ahead of the game is in installation friendliness, as well as add-on management (installation and uninstallation of components, modules, etc.)
In comparison, Drupal requires one to manually unzip add-ons on the server, possibly create the necessary SQL tables from supplied scripts, and there is no friendly installation “wizard” to guide you through first-time installation. (This is all evolving, though; it shouldn’t be many months before there is a comprehensive installation system available in Drupal.)
If the above points are very important considerations to you, as they certainly might be to less technically-savvy users, then you may not really benefit from this article. As stated, our team hails from a quite different user segment.

And I completely agree with that. Also, Joomla has thousands of third-party components and templates (much more than Drupal has). That’s the first reason why I’m in love with Joomla.

I hope you know what is Google Trends. Shortly: it’s a tool that analyzes Google web searches and that can visualize search trends over time using so called “search-volume graphs”. These graphs usually provide a good mechanism to compare the popularity of two or more products. Buytaert.net provides us with the search-volume graph that compares Drupal and Joomla:

Graph

It is worth pointing out that Joomla has been around a lot longer than the graph suggests. In 2005, the bulk of Mambo’s core developers left Mambo and started Joomla after a dispute with Miro Corporation, the company that founded Mambo. Keep this in mind when interpreting the graph. (I tried adding Mambo to the graph but the term Mambo isn’t unique to Mambo, the content management system.)
That said, you can see that Joomla is more popular than Drupal, and that Joomla has been growing a lot faster. Why? The general consensus is that Joomla has a more appealing balance between functionality, flexibility, performance, quality of code, ease of use, documentation, user interface design, support and product marketing.

And that’s the second reason why I recommend Joomla – it’s popular, it has very… very big community so all bugs are found and fixed quickly and number of different add-ons for that CMS grows extremely fast.
You might have heard Mambo* is “bulky, badly optimized for search engines, and generally rigid and brittle to customized. Drupal, on the other hand, is perhaps the most search engine friendly CMS on the market. Its modular, flexible, its underlying design has been guided by a stellar philosophy” (http://www.nicklewis.org/mambo-vs-drupal) . But Mambo is not a Joomla (check Nick’s comment below).

*There was “Joomla” before Nov 15, thanks to Nick Lewis for correction.

Now, to be honest, I’d like to tell you there is one thing why I can recommend you to use Drupal. It’s coded better than Joomla. It’s a bit faster than Joomla. Buytaert.net says Drupal is 319% faster than Joomla (wow!) but that’s also not a reality. To check this I took my old PC (P-133Mhz) installed there LAMP (Linux+Apache+MySQL+PHP), set up Joomla and Drupal, modified code a bit to get perfomance data and after all of that I got interesting things – Joomla and Drupal have shown comparable time. Drupal was faster only 1%-3%. Old PC demonstrated server’s behaviour at high loads, so … that’s the third reason why I still love Joomla.

And, to get this finished, here’s what users wrote at Yahoo! Answers:

There are many Open Source CMS applications and as usual choosing an appropriate one is difficult. In my experience, I find Joomla and Mambo the best but I prefer Joomla as it is being developed more at present as it is an off-shoot of Mambo.
Drupal looks impressive but I found it hard to configure. We used another one called Midgard for a project but there has been little development of that CMS plus it is hard to configure.
The reasons why I like Joomla better are as follows:
1. Joomla has many extensions (add-ons) so it can be scaled easily - http://extensions.joomla.org
2. There are many free as well as paid templates available.
3. Joomla is search engine friendly and this is an important consideration while marketing your website on the Internet. There is as an Search Engine Friendly (SEF) extension at http://open-sef.org/
4. Most hosting providers have Joomla that can be installed through their Control Panels.

So, if you are a hard programmer, you like to experiment with code, you don’t care much about user-friendly and usability or you think Drupal’s solution is OK – use Drupal.

If you care about your customers, you wish to ba able to set up addons (components and modules) with 2-3 mouse clicks without even unpacking files manually on server (Joomla will do that for you) or you’re just like me love Joomla – go ahead and give it a try.

Added on Nov, 16th: Guys, I’m not the only who respect Joomla. PackTrub admitted it to be the best. Wow!

Alexander Alfimov. Nov, 09 2006

Posted by Posted by Alexander Alfimov under Filed under CMS, Joomla Comments 15 Comments »

20th Nov 2006

About Blog

Hi!

I’m not sure how did you find this blog, googled or clicked a link somewhere, but I’m really glad to see you here. This is the place where I try to put all my thoughts together into posts. Hope you’ll find at least one sentence interesting. Please don’t hesitate to leave comments if you wish.

Best regards, Alexander Alfimov

Posted by Posted by Alexander Alfimov under Filed under CMS Comments No Comments »

Close
E-mail It