[FASTCGI] very low performance - explained

Igor Franchuk sprog at online.ru
Tue Jan 26 19:37:02 EST 2010


Hello Тарасов,

Tuesday, January 26, 2010, 9:07:42 PM, you wrote:

I don't know haskell either. But I bet you have no FastCGI loop.
It's a classic mistake leading to performance downgrade.

http://mult.ifario.us/p/wiring-haskell-into-a-fastcgi-web-server
http://hackage.haskell.org/packages/archive/fastcgi/3001.0.2.2/doc/html/Network-FastCGI.html

FastCGI is made for performance, still one can easily downgrade with
FastCGI like you did in your example below.

In your particular task you should not read the file, instead you should just
output:

setHeader "Location: ./pic.jpg"

to make Apache to do streaming. Not to mention that cgi-bin is not for
images and most likely out-of-the-box Location: would fail. Yet, the
way you handle images is not the cause of your problems, the cause is most
likely the absence of CGI Loop. I'm not sure about the loop, still to make
things clear I will give you some information to think about.

FastCGI is a great technology that makes your application to
function almost with the speed of Apache module with _low_ costs to
achieve it. That technology gives power to the languages which are
unable to integrate with Apache on module level. The tragic but not
officially recognized failure of a famous Apache module extension
like mod_php which proved that FastCGI is a better way to do fast
things than pursing mod_language approach which has many many problems.
It came to the point now that most of the hosters are running PHP as
FastCGI and not as mod_php by various reason, the main reason, of
course, the way PHP consumes the server's resources. Instead of consuming
resources on demand, which pre PHP languages were using (like Perl),
PHP pursuits "load all GNU libraries which I could port to function
within PHP core" approach which looks cool to a noob user which can do
just about anything, the question remains - do we need to do anything
slow or fast. The PHP team though it's going to be fast. The PHP came
to the world as a Fast alternative to Perl and, that speed came out of
integration with Apache. PHP was always web oriented and many things
were easier with it.

As a practical example, out of the box PHP running as a mod_php in the
real world would eat about 70Mb of memory, to be compared with a
similar C++ program, running in FastCGI mode that would consume only 3Mb,
with 3-4 times drop in processor usage.

The performance gains are achieved by the ability of FastCGI to
run your app as a daemon or process. Once loaded, your application
is focused on processing the requests.

Focusing on important things, a CGI application is working like:

1. Apache got request GET /your_script

2. Your CGI app is loaded, in case of say Perl it's compiled first then
   loaded. To compile/translate it, the perl would have to be started.
   When it starts it loads dynamic libraries, which when loaded are pulling
   other dynamic libraries (in most cases).

3. When all is settled with Perl, your script is translated.

4. Now we're ready to execute our script.

5. Your script pulls the data out of environment, does it's job
   and calls exit.
   
6. The memory used by perl is freed, your compiled code is wasted.
   the libraries are unloaded and now we're waiting for step 1, just
   to make another similar job.
   
-----------------------------------------------------------------------
In fast CGI Mode your application works differently:

User starts /your_script

1. Apache got request GET /your_script

2. Your CGI app is loaded, in case of say Perl it's compiled first then
   loaded. To compile/translate it, the perl would have to be started.
   When it starts it loads dynamic libraries, which when loaded, are pulling
   other dynamic libraries (in most cases).

3. When all is settled with Perl, your script is translated.

4. Now we're ready to execute our script.

5. Your script pulls the data out of environment, does
   it's job and it goes Back to step 4


You see, your fast cgi app just stays in the memory, loaded, armed and
ready to process another request. When next request comes in, there
are no steps 1,2,3,4,6. "Back to step 4" is a FasctCGI loop which should
be programmed into your FastCGI app, in case of PHP that fastcgi loops
stays outside of the user code yet in most languages it's programmed
by user. In PHP you can't do "slow jobs" before entering FastCGI loop,
so even in FastCGI mode php will fail to achieve the same performance
as even Perl in FastCGI mode.

So, what you're doing is making a FastCGI app to run thorough steps
1-6 over and over again, instead of Going back to step 4 you're
calling EXIT, which makes your app to behave exactly like a normal
CGI. And, unlike a normal CGI app, a FastCGI app has an overhead in
starting a FasctCGI server for the first time and in stopping it.

Since you've done very badly in programming your first FastCGI app,
you're making the server to run through steps 1 to 6 PLUS FastCGI
overhead which in the first time is significant. That's why
you have such a bad performance out of a sound technology.



Now to haskel, taking my fist glance at haskell:

import Control.Concurrent
import System.Posix.Process (getProcessID)

import Network.FastCGI

test :: CGI CGIResult
test = do setHeader "Content-type" "text/plain"
          pid <- liftIO getProcessID
          threadId <- liftIO myThreadId
          let tid = concat $ drop 1 $ words $ show threadId
          output $ unlines [ "Process ID: " ++ show pid,
                             "Thread ID:  " ++ tid]


                             
main = runFastCGIConcurrent forkIO 10 test

^^^^^^^^^^^^^^^^ I would say that this is the FastCGI loop

I may be mistaking but it looks like

runFastCGIConcurrent would use forkIO to run test procedure with concurrent
model.



import Control.Concurrent
import System.Posix.Process (getProcessID)

import Network.FastCGI

test :: CGI CGIResult
test = do setHeader "Content-type" "text/plain"
          pid <- liftIO getProcessID
          threadId <- liftIO myThreadId
          let tid = concat $ drop 1 $ words $ show threadId
          output $ unlines [ "Hello, I'm Haskell what a purely logical language I am",
                             "With a good concurrency too, now featuring FastCGI, ",
                             "I've run out of +, they are all used for mathematics, and the point came to the",
                             "complex floating point ops, so I'm kind of in need",
                             "if you close your eyes you may imagine",
                             "me as one huge but beautiful command line",
                             "but I'm not a command line I'm equation",
                             "of course I'm not as half as alien as Perl might be",
                             "I am a gift to the scientists all around the world",
                             "they now can have equations as long as they need",
                             "Process ID: " ++ show pid,
                             "Thread ID:  " ++ tid]

init = do
//YOU DO ALL THE HEAVY STUFF HERE, LIKE DB CONNECTIONS,
//INITIALIZATIONS, HASHING, B-TREES, ANYTHING, BEFORE ENTERING FASTCGI LOOP
//



// now we're entering FastCGI loop where we will reside till Apache
// server is working and we do not exit on purpose or abnormally

main = init runFastCGIConcurrent forkIO 10 test



ТЕ> Thanks for your answers!

>> How did you write the code to read the file?
>> What language did you write it in? You haven't provided enough information.

ТЕ> I wrote it in haskell, just reading file and writting it out using ByteString:

ТЕ> import Network.FastCGI
ТЕ> import qualified Data.ByteString.Lazy as BL

ТЕ> main = runFastCGIConcurrent 1000 (handleErrors cgiMain)

ТЕ> cgiMain :: CGI CGIResult
ТЕ> cgiMain = do
ТЕ>     setHeader "Content-type" "image/jpeg"
ТЕ>     b <- liftIO $ BL.readFile "/usr/lib/cgi-bin/pic.jpg"
ТЕ>     outputFPS b

>> FastCGI is not particularly appropriate for this task.  Apache is most
>> likely much more efficient at the fairly simple task of streaming files via
>> HTTP, and doesn't incur the overhead that FastCGI does in performing that
>> task.

ТЕ> I am planning to write complex content management system with storing pictures in dedicated storage, that's why I have to output pictures through the
ТЕ> application.

>> FastCGI is more appropriate for tasks that you would otherwise write a CGI
>> program to do, and should be faster in those cases.  It is also appropriate
>> for tasks where you would consider loading mod_php, mod_perl, etc. into
>> your Apache configuration; in those cases, the performance difference will
>> probably be negligible, but you have a lot more control over the design in
>> terms of load-balancing and security.

ТЕ> So, I will use something different than fastcgi.
ТЕ> _______________________________________________
ТЕ> FastCGI-developers mailing list
ТЕ> FastCGI-developers at mailman.fastcgi.com
ТЕ> http://mailman.pins.net/mailman/listinfo.cgi/fastcgi-developers



-- 
www.rol.ru
Best regards,
 Igor                            mailto:sprog at online.ru



More information about the FastCGI-developers mailing list