maximum number of open sockets, files, threads, etc..?

http://www.dslreports.com/forum/r20368916-maximum-number-of-open-sockets-files-threads-etc
http://stackoverflow.com/questions/2185834/maximum-number-of-socket-in-java

Send Eventletdev mailing list submissions to
       [e-mail].<wbr></wbr>com

To subscribe or unsubscribe via the World Wide Web, visit
       https://lists.secondlife.com/<wbr></wbr>cgi-bin/mailman/listinfo/<wbr></wbr>eventletdev
or, via email, send a message with subject or body 'help' to
       eventletdev-request@lists.<wbr></wbr>secondlife.com

You can reach the person managing the list at
       eventletdev-owner@lists.<wbr></wbr>secondlife.com

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Eventletdev digest..."


Today's Topics:

  1. Re: rollback (Ryan Williams (Which))
  2. Re: rollback (Sergey Shepelev)
  3. Re: rollback (Ryan Williams (Which))


------------------------------

<wbr></wbr>------------------------------<wbr></wbr>----------

Message: 1
Date: Mon, 01 Mar 2010 21:18:41 -0800
From: "Ryan Williams (Which)" <[e-mail]>
Subject: Re: [Eventletdev] rollback
To: Amir Yalon <[e-mail]>
Cc: eventlet <[e-mail].<wbr></wbr>com>
Message-ID: <4B8C9FB1.6040903@lindenlab.<wbr></wbr>com>
Content-Type: text/plain; charset=UTF-8; format=flowed

On 3/1/10 9:00 PM, Amir Yalon wrote:
> Hi,
>
> On Sat, Feb 13, 2010 at 08:20, Ryan Williams (Which) <[e-mail]
> <mailto:[e-mail]>> wrote:
>
>     def serve(sock, handle, concurrency=1000):
>
>          The value in *concurrency* controls the maximum number of
>          greenthreads that will be open at any time handling requests.  When
>          the server hits the concurrency limit, it stops accepting new
>          connections until the existing ones complete.
>     """
>          pass
>
>
> Being able to limit the number of concurrent handlers is a reasonable
> requirement, but why do you want to force such a small value by default?
> What is the bottleneck here, actually?
>
> As I see it, the whole idea of greenlet, Stackless etc. is to enable
> concurrency in the hundreds of thousands on common hardware.

Yeah, that's a reasonable point; I just picked essentially a random
round number.  My experience is that my own apps have gotten CPU-bound
quickly with actual work and so can't meaningfully use more than about
1000 concurrently. (tagline for Eventlet: "it helps you get to 100% CPU
utilization!")  I probably have pretty CPU-bound workloads though, and
should probably bump that number up.  What about 10000?

Thoughts from other folks on what concurrencies they use?


------------------------------

Message: 2
Date: Tue, 2 Mar 2010 09:22:40 +0300
From: Sergey Shepelev <[e-mail]>
Subject: Re: [Eventletdev] rollback
To: "Ryan Williams (Which)" <[e-mail]>
Cc: eventlet <[e-mail].<wbr></wbr>com>
Message-ID:
       <2d8fb9951003012222s282b2246m5<wbr></wbr>[e-mail]<wbr></wbr>>
Content-Type: text/plain; charset=UTF-8

On Tue, Mar 2, 2010 at 8:18 AM, Ryan Williams (Which) <[e-mail]> wrote:
> On 3/1/10 9:00 PM, Amir Yalon wrote:
>> Hi,
>>
>> On Sat, Feb 13, 2010 at 08:20, Ryan Williams (Which) <[e-mail]
>> <mailto:[e-mail]>> wrote:
>>
>> ? ? def serve(sock, handle, concurrency=1000):
>>
>> ? ? ? ? ?The value in *concurrency* controls the maximum number of
>> ? ? ? ? ?greenthreads that will be open at any time handling requests. ?When
>> ? ? ? ? ?the server hits the concurrency limit, it stops accepting new
>> ? ? ? ? ?connections until the existing ones complete.
>> ? ? """
>> ? ? ? ? ?pass
>>
>>
>> Being able to limit the number of concurrent handlers is a reasonable
>> requirement, but why do you want to force such a small value by default?
>> What is the bottleneck here, actually?
>>
>> As I see it, the whole idea of greenlet, Stackless etc. is to enable
>> concurrency in the hundreds of thousands on common hardware.
>
> Yeah, that's a reasonable point; I just picked essentially a random
> round number. ?My experience is that my own apps have gotten CPU-bound
> quickly with actual work and so can't meaningfully use more than about
> 1000 concurrently. (tagline for Eventlet: "it helps you get to 100% CPU
> utilization!") ?I probably have pretty CPU-bound workloads though, and
> should probably bump that number up. ?What about 10000?
>
> Thoughts from other folks on what concurrencies they use?

1. (that's why i don't like default values: people tend to think they
are somehow "proper") Man, it's an argument to the function. Just
forget that it has a default value and pass your favourite 300000
there. You only have 64K of sockets so enjoy IOError in the middle.

2. (about the number)
The guy forgot that serve() is about network connections. greenlet is
about 100-s of thousands *threads*. And it does handle that many. The
proposal to increase default size would actually make sense for
`GreenPool`, not for `serve`.

You have to do a lot of job to handle >20K of connections on a single
box. Because each connection take perceptible amount of system
resources, both RAM and CPU wise. Igor Sysoev (nginx author) once
posted a sample nginx configuration (along with highly tuned FreeBSD
kernel configuration) that sustain 9K active req/s along with 45K
passive keep-alive connections. So sum is 54K connections. This is
about an extremely optimized web server in C!

3. Sure, bumping default limit up to 10K would be a fun pun to C10K
problem [1]. :)
Leaving it same 1K or removing the default value ? all make equal
sense to me ? it's just not relevant so much as long as it's not a
constant but a setting. There are more important things.

[1] http://www.kegel.com/c10k.html

P.S.: list still is not configured properly (to add Reply-To header)
so people still send you private e-mails, right? ;)

> ______________________________<wbr></wbr>_________________
> Click here to unsubscribe or manage your list subscription:
> https://lists.secondlife.com/<wbr></wbr>cgi-bin/mailman/listinfo/<wbr></wbr>eventletdev
>


------------------------------

Message: 3
Date: Tue, 02 Mar 2010 00:08:15 -0800
From: "Ryan Williams (Which)" <[e-mail]>
Subject: Re: [Eventletdev] rollback
To: Sergey Shepelev <[e-mail]>
Cc: eventlet <[e-mail].<wbr></wbr>com>
Message-ID: <4B8CC76F.2000701@lindenlab.<wbr></wbr>com>
Content-Type: text/plain; charset=UTF-8; format=flowed

On 3/1/10 10:22 PM, Sergey Shepelev wrote:
> 3. Sure, bumping default limit up to 10K would be a fun pun to C10K
> problem [1]. :)
> Leaving it same 1K or removing the default value ? all make equal
> sense to me ? it's just not relevant so much as long as it's not a
> constant but a setting. There are more important things.

Yeah, my thoughts exactly on the pun.

> [1] http://www.kegel.com/c10k.html
>
> P.S.: list still is not configured properly (to add Reply-To header)
> so people still send you private e-mails, right? ;)

I prefer it that way because accidentally sending personal email on-list
is way worse than having to re-send if you forget to include the list.


------------------------------

______________________________<wbr></wbr>_________________
Eventletdev mailing list
[e-mail].<wbr></wbr>com
https://lists.secondlife.com/<wbr></wbr>cgi-bin/mailman/listinfo/<wbr></wbr>eventletdev


End of Eventletdev Digest, Vol 27, Issue 1
******************************<wbr></wbr>************