Joomla Forum

Our site was using ncaster and phpbb when we decided to change to joomla-smf.

All the transfering and revamp took 1 month. When we deploy it into our dedicated webserver, things start to go wrong. We see our server load going so high until server stop responding.

We manage to trace the problem to joomla-smf 2.0 rc1.

When we disable SMF, the server will be running fine. Once we enable SMF, when user login from joomla and redirected to forum, server load will become super high where we see a few httpd processes running 80-90mb.

If we were to login to SMF directly without going through joomla, server will be fine.

From my understanding, once you login to joomla, joomla will create one user session and once redirected to SMF, the bridge will create another session in SMF so there will be two sessions running together. Could that be the cause the resource went up?

More information about my server :

Dedicated Server P4 2.8c
1 GB RAM
HDD space (enough)

WHM 10.8.0 cPanel 10.9.0-R34
Apache 1.3.x
PHP 4.4.4
mysql 4.1.12
mailscanner

joomla 1.0.11
SMF 1.1rc3
joomla-smf 2.0 rc1
cbuilder 1.0.1 from joomlapolis

We tested the forum as a standalone and it's working fine. We have around 100,000 members in our forum.

Please help....

edit :
we already tried most of the SMF optimization steps.

1) optimize mysql, optimize apache
2) eaccelerator
3) change some tables to innodb

The bridge is using caching and sessions to improve performance.  So by default memory usage will be higher however  I never saw anything even close to 80-90MB though.  Although you have to realize the load on the server is double what either SMF or Joomla alone would place because you are loading them both simultaneously. 

Do you have eaccelerator caching to memory as well?  or to disk?

helo, yeah..

we did install the eaccelerator and enable level 1 cache in smf.

I don't know what level 1 is.  But if it is caching to memory then that might explain the large memory usage, especially if Joomla caching is on. 

Eventhought if the cache were off in Joomla and SMF, the CPU usage was high as well.

Just wondering, if i were to login via joomla, joomla will create a session for me while i m still only browsing in joomla. So i presume at this stage, no SMF session was created until i enter the forum, am i right?

no, your smf session is created at the same time as your joomla session. 

any idea which angle should be tackle this problem? We have tried all sorts of things to no avail. The problem is still this httpd process going 80mb once someone go into the forum through joomla.

This is how it goes.

1) Go into main page (which is joomla)

2) click on forum button

3) now in SMF and then logon

4) start using forum

5) in the background...server load starts going up up up and away.....

are you saying there is a memory leak or are you saying it consumes a larger amount of server resources?  Have you tried disabling eAccelerator or setting it to disk only?  Have you disabled joomla caching?

I'm going to look at reworking some of the session storage stuff to see if I can cache one instance of it for everyone rather than one per user.  This will probably help your situation. 

I don't know whether it's memory leak or not. How to know anyway?

I just see using top. Once I tried accessing the forum through joomla, I will see cpu load going high and httpd processes running 90mb, then more and more httpd processes running at 90mb. IF the user continue using the forum browsing, searching, posting and so on, the system load will go higher and higher using load value of more than 120...soon server stops responding.

I already tried with/without eaccelerator, cache and so on.

I am now trying with a new setup of plain joomla, smf and bridge with same database to see if problem exist. Also, maybe it's cbuilder causing this.

well, what I am doing is storing a list of links, so its only text.  At absolute maximum a single user's session would have maybe 1MB.  So I can't se how a single process would hit 80 or 90 ever.  But I am going to rework this, I think I can store all the links in cache but store one copy for everyone instead of one copy per user.  My initial reason for not doing this is that I thought the links would get crossed for profiles etc.  Now I feel pretty confident they won't.  I plan on working on this this weekend.

ok, i have did some test on the user tables.

I did a fresh install on joomla, CB and the bridge. with only one user, everything seems to be working fine. So i copied that large user table over and login, the resource usage went up high back.

Just wondering, could it be possible the extra fields which i have added in CB that is causing the issue?

PS : regarding the bridge.
1. if you update your username in CB, it will only update the JOS_USER and JOS_COMPROFILEr but not the SMF_MEMBERS
2. if you were to delete a member via CB, the respective record in  jos_core_acl_aro, jos_core_acl_groups_aro_map and smf_members were not removed.

do you have the cb plugin installed?  It doesn't sound like it.

So without traffic on the system(meaning no users except you) this is occurring?  If that is the case, then I am not sure how this part of the bridge can cause this because this session storage only occurs when a session is started meaning someone visits or logs in to the site.  Are you using something to simulate that?

new findings.

We cut down the users to half of 100,000. Which is around 45k. We try to duplicate the problem and this time httpd process showing usage of around 40-60mb. This shows that the memory usage scales with the number of members in the DB.

So does that means what you (wolverine) said of each member having their own cache takes a toll in httpd memory usage as well? How about someone with large user base provide some comment?

in additon to that, is only being access by one person. if there were 2nd person accesing, you notice that there will be another httpd with 40-60mb. At first we thought of adding rams to 2Gig should solve the problem, but at this point it seems is not going to solve the issue, everytime if a person access there will be another httpd pop up with high mem.

I have completely reworked this and now everything is cached instead of stored in each users session.  This feature will be included in JSMF 2.0RC2.

Hi Wolverine,

Any idea when RC2 will be released or can I replace the existing code with the reworked code?

Thanks.

RC2 was given to the test group today.  After things seem stable I'll release it to the rest of you all.  ;D