[GENERAL] 8.2.3: Server crashes on Windows using Eclipse/Junit

[Magnus Hagander]

Maybe we should put an #ifdef WIN32 into guc.c to limit max_connections
to something we know the platform can stand? It'd be more comfortable
if we understood exactly where the limit was, but I think I'd rather
have an "I'm sorry Dave, I can't do that" than random-seeming crashes.

Yeayh, that's probably a good idea - except we never managed to figure out
where the limit is. It appears to vary pretty wildly between different
machines, for reasons we don't really know why (total RAM has some effect
on it, but that's not the only one, for example)

I tried generating idle connections in an effort to reproduce
Laurent's problem, but I ran into a local limit instead: for each
backend, postmaster creates a thread and burns 4MB of its 2GB address
space. It fails around 490.

Oh, that's interesting. That's actually a sideeffect of us increasing
the stack size for the postgres.exe executable in order to work on other
things. By default, it burns 1MB/thread, but ours will do 4MB. Never
really thought of the problem that it'll run out of address space.
Unfortunately, that size can't be changed in the CreateThread() call -
only the initially committed size can be changed there.

There are two ways to get around it - one is not using a thread for each
backend, but a single thread that handles them all and then some sync
objects around it. We originally considered this but said we won't
bother changing it because the current way is simpler, and the overhead
of a thread is tiny compared to a process. I don't think anybody even
thought about the fact that it'd run you out of address space...

The other way is to finish off win64 support :-) Which I plan to look
at, but I don't think that alone should be considered a solution.

The question is if it's worth fixing that part, if it will just fall
down for other reasons before we reach these 500 connections anyway. Can
you try having your program actually run some queries and so, and not
just do a PQconnect? To see if it falls over then, because it's been
doing more?

Laurent's issue must depend on other load characteristics. It's
possible to get a trace of DLL loads, but I haven't found a
noninvasive way of doing that. It seems to require a debugger be
attached.

AFAIK, it does require that, yes.

//Magnus

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

[Trevor Talbot]

Post by Magnus Hagander

I tried generating idle connections in an effort to reproduce
Laurent's problem, but I ran into a local limit instead: for each
backend, postmaster creates a thread and burns 4MB of its 2GB address
space. It fails around 490.

Oh, that's interesting. That's actually a sideeffect of us increasing
the stack size for the postgres.exe executable in order to work on other
things. By default, it burns 1MB/thread, but ours will do 4MB. Never
really thought of the problem that it'll run out of address space.
Unfortunately, that size can't be changed in the CreateThread() call -
only the initially committed size can be changed there.
There are two ways to get around it - one is not using a thread for each
backend, but a single thread that handles them all and then some sync
objects around it. We originally considered this but said we won't
bother changing it because the current way is simpler, and the overhead
of a thread is tiny compared to a process. I don't think anybody even
thought about the fact that it'd run you out of address space...

I'd probably take the approach of combining win32_waitpid() and
threads. You'd end up with 1 thread per 64 backends; when something
interesting happens the thread could push the info onto a queue, which
the new win32_waitpid() would check. Use APCs to add new backends to
threads with free slots.

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org/

[Magnus Hagander]

Post by Trevor Talbot

Post by Magnus Hagander

I tried generating idle connections in an effort to reproduce
Laurent's problem, but I ran into a local limit instead: for each
backend, postmaster creates a thread and burns 4MB of its 2GB address
space. It fails around 490.

Oh, that's interesting. That's actually a sideeffect of us increasing
the stack size for the postgres.exe executable in order to work on other
things. By default, it burns 1MB/thread, but ours will do 4MB. Never
really thought of the problem that it'll run out of address space.
Unfortunately, that size can't be changed in the CreateThread() call -
only the initially committed size can be changed there.
There are two ways to get around it - one is not using a thread for each
backend, but a single thread that handles them all and then some sync
objects around it. We originally considered this but said we won't
bother changing it because the current way is simpler, and the overhead
of a thread is tiny compared to a process. I don't think anybody even
thought about the fact that it'd run you out of address space...

I'd probably take the approach of combining win32_waitpid() and
threads. You'd end up with 1 thread per 64 backends; when something
interesting happens the thread could push the info onto a queue, which
the new win32_waitpid() would check. Use APCs to add new backends to
threads with free slots.

I was planning to make it even easier and let Windows do the job for us,
just using RegisterWaitForSingleObject(). Does the same - one thread per
64 backends, but we don't have to deal with the queueing ourselves.
Should be rather trivial to do.

Keeps win32_waitpid() unchanged.

That said, refactoring win32_waitpid() to be based on a queue might be a
good idea *anyway*. Have the callback from above put something in the
queue, and go with your idea for the rest.

//Magnus

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ***@postgresql.org so that your
message can get through to the mailing list cleanly

[Trevor Talbot]

Post by Magnus Hagander

Post by Trevor Talbot
I'd probably take the approach of combining win32_waitpid() and
threads. You'd end up with 1 thread per 64 backends; when something
interesting happens the thread could push the info onto a queue, which
the new win32_waitpid() would check. Use APCs to add new backends to
threads with free slots.

I was planning to make it even easier and let Windows do the job for us,
just using RegisterWaitForSingleObject(). Does the same - one thread per
64 backends, but we don't have to deal with the queueing ourselves.

Oh, good call -- I keep forgetting the native thread pool exists.

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

[Magnus Hagander]

Post by Trevor Talbot

Post by Magnus Hagander

Post by Trevor Talbot
I'd probably take the approach of combining win32_waitpid() and
threads. You'd end up with 1 thread per 64 backends; when something
interesting happens the thread could push the info onto a queue, which
the new win32_waitpid() would check. Use APCs to add new backends to
threads with free slots.

I was planning to make it even easier and let Windows do the job for us,
just using RegisterWaitForSingleObject(). Does the same - one thread per
64 backends, but we don't have to deal with the queueing ourselves.

Oh, good call -- I keep forgetting the native thread pool exists.

Taking this one to -hackers once and for all now...

Can you try the attached patch? See how many backends you can get up to.

This patch changes from using a single thread for each backend started to
using the builtin threadpool functionality. It also replaces the pid/handle
arrays with an i/o completion port. The net result is also, imho, much more
readable code :-)

Beware - there's still plenty of debugging code in there :-)

//Magnus

[Trevor Talbot]

Post by Magnus Hagander
Can you try the attached patch? See how many backends you can get up to.
This patch changes from using a single thread for each backend started to
using the builtin threadpool functionality. It also replaces the pid/handle
arrays with an i/o completion port. The net result is also, imho, much more
readable code :-)

The patch looks good; I'm not set up to build yet, but I should be
able to test it sometime in the next week.

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ***@postgresql.org so that your
message can get through to the mailing list cleanly

[Magnus Hagander]

Post by Trevor Talbot

Post by Magnus Hagander
Can you try the attached patch? See how many backends you can get up to.
This patch changes from using a single thread for each backend started to
using the builtin threadpool functionality. It also replaces the pid/handle
arrays with an i/o completion port. The net result is also, imho, much more
readable code :-)

The patch looks good; I'm not set up to build yet, but I should be
able to test it sometime in the next week.

I've uploaded a set of binary files to
http://www.hagander.net/pgsql/pgsql_83_snapshot_win32child.zip.
You'll need to get the dependency DLLs elsewhere, but you may have them
already.

//Magnus


---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

[Trevor Talbot]

Post by Trevor Talbot

Post by Magnus Hagander
Can you try the attached patch? See how many backends you can get up to.
This patch changes from using a single thread for each backend started to
using the builtin threadpool functionality. It also replaces the pid/handle
arrays with an i/o completion port. The net result is also, imho, much more
readable code :-)

The patch looks good; I'm not set up to build yet, but I should be
able to test it sometime in the next week.

Sorry about the long delay; I retested with the 8.3-beta2 installer,
still Win2003 SP2 32bit.

I stopped the test at 824 connections because I was about to run out
of memory (1.25GB RAM + 3.75GB swap), but postmaster VM space usage
was only 191MB.

As for desktop heap, only 65KB of the service heap was allocated, or
about 80 bytes per connection. No danger of hitting limits in the
kernel memory pools either.

Available RAM seems like a pretty reasonable limit to me ;)

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

[Magnus Hagander]

Post by Trevor Talbot

Post by Trevor Talbot

Post by Magnus Hagander
Can you try the attached patch? See how many backends you can get up to.
This patch changes from using a single thread for each backend started to
using the builtin threadpool functionality. It also replaces the pid/handle
arrays with an i/o completion port. The net result is also, imho, much more
readable code :-)

The patch looks good; I'm not set up to build yet, but I should be
able to test it sometime in the next week.

Sorry about the long delay; I retested with the 8.3-beta2 installer,
still Win2003 SP2 32bit.
I stopped the test at 824 connections because I was about to run out
of memory (1.25GB RAM + 3.75GB swap), but postmaster VM space usage
was only 191MB.

Great.
I'm thinking this change may be big enough to actually backport to 8.2 -
what to others feel about that?

Assuming it is, I still think we should wait at least until we've run 8.3
RC for a while - probably until 8.3 has been actually released and run for
a while, to make sure we have a *lot* of testing of it before we consider
backpatching.

Post by Trevor Talbot
As for desktop heap, only 65KB of the service heap was allocated, or
about 80 bytes per connection. No danger of hitting limits in the
kernel memory pools either.

As Dave said, it could be that the server version uses a lot less heap per
process, which would be another good reason to use server rather than XP to
run postgresql. But might there also be other differences, such as some
third party (or non-core microsoft) product installed?

Dave, on your XP test, was that on a clean XP with nothing like AV or any
3rd party stuff on it?

Post by Trevor Talbot
Available RAM seems like a pretty reasonable limit to me ;)

Yeah, not much we can do about that one :-)

//Magnus

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

[Dave Page]

Post by Magnus Hagander

Post by Trevor Talbot
As for desktop heap, only 65KB of the service heap was allocated, or
about 80 bytes per connection. No danger of hitting limits in the
kernel memory pools either.

As Dave said, it could be that the server version uses a lot less heap per
process, which would be another good reason to use server rather than XP to
run postgresql. But might there also be other differences, such as some
third party (or non-core microsoft) product installed?
Dave, on your XP test, was that on a clean XP with nothing like AV or any
3rd party stuff on it?

No, it was on my XP laptop which runs Sophos AV. I'm not convinced it's
AV related though - in my test code I proved pretty conclusively that
just initialising user32.dll ate the desktop heap.

/D

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ***@postgresql.org so that your
message can get through to the mailing list cleanly

[Magnus Hagander]

Post by Dave Page

Post by Magnus Hagander

Post by Trevor Talbot
As for desktop heap, only 65KB of the service heap was allocated, or
about 80 bytes per connection. No danger of hitting limits in the
kernel memory pools either.

As Dave said, it could be that the server version uses a lot less heap per
process, which would be another good reason to use server rather than XP to
run postgresql. But might there also be other differences, such as some
third party (or non-core microsoft) product installed?
Dave, on your XP test, was that on a clean XP with nothing like AV or any
3rd party stuff on it?

No, it was on my XP laptop which runs Sophos AV. I'm not convinced it's
AV related though - in my test code I proved pretty conclusively that
just initialising user32.dll ate the desktop heap.

I'm certainly not convinved about that either, but we should make a test on
a VM at some point.

Sophos AV has plugins into for example the explorer (I assume - most AV
does, haven't used Sophos specifically myself), which may be done with
extra DLLs loading along with user32.dll (runtime linked) or something like
that. I just want to be sure we exclude that possibility.

//Magnus

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

[Dave Page]

Post by Magnus Hagander
I'm certainly not convinved about that either, but we should make a test on
a VM at some point.
Sophos AV has plugins into for example the explorer (I assume - most AV
does, haven't used Sophos specifically myself), which may be done with
extra DLLs loading along with user32.dll (runtime linked) or something like
that. I just want to be sure we exclude that possibility.

Yeah, iirc it does. I don't have time for that at the moment, but I can
fire you a copy of my test code if you do.

/D

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ***@postgresql.org so that your
message can get through to the mailing list cleanly

[Trevor Talbot]

Post by Magnus Hagander

Post by Trevor Talbot
As for desktop heap, only 65KB of the service heap was allocated, or
about 80 bytes per connection. No danger of hitting limits in the
kernel memory pools either.

As Dave said, it could be that the server version uses a lot less heap per
process, which would be another good reason to use server rather than XP to
run postgresql. But might there also be other differences, such as some
third party (or non-core microsoft) product installed?

The XP SP2 machine I tried 8.2.5 on was chewing up about 3.1KB per
process, and it's not running anything invasive (AV or otherwise).

I've been trying to find out exactly what's in the desktop heap, but I
haven't had much luck so far. Apparently Microsoft changed the
implementation after Win2000, and didn't bother teaching the public
debugging tools about it. The details just don't seem to exist
anymore :(

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

[Magnus Hagander]

Post by Trevor Talbot

Post by Magnus Hagander

Post by Trevor Talbot
As for desktop heap, only 65KB of the service heap was allocated, or
about 80 bytes per connection. No danger of hitting limits in the
kernel memory pools either.

As Dave said, it could be that the server version uses a lot less heap per
process, which would be another good reason to use server rather than XP to
run postgresql. But might there also be other differences, such as some
third party (or non-core microsoft) product installed?

The XP SP2 machine I tried 8.2.5 on was chewing up about 3.1KB per
process, and it's not running anything invasive (AV or otherwise).

Then I think we can claim that Server is just better than Workstation in
this regard. Maybe we should put that in the FAQ?

Post by Trevor Talbot
I've been trying to find out exactly what's in the desktop heap, but I
haven't had much luck so far. Apparently Microsoft changed the
implementation after Win2000, and didn't bother teaching the public
debugging tools about it. The details just don't seem to exist
anymore :(

Yeah, there are very little docs at all about the desktop heap AFAICT.

//Magnus

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

[Trevor Talbot]

Post by Magnus Hagander

Post by Trevor Talbot

Post by Magnus Hagander

Post by Trevor Talbot
As for desktop heap, only 65KB of the service heap was allocated, or
about 80 bytes per connection. No danger of hitting limits in the
kernel memory pools either.

As Dave said, it could be that the server version uses a lot less heap per
process, which would be another good reason to use server rather than XP to
run postgresql. But might there also be other differences, such as some
third party (or non-core microsoft) product installed?

The XP SP2 machine I tried 8.2.5 on was chewing up about 3.1KB per
process, and it's not running anything invasive (AV or otherwise).

Then I think we can claim that Server is just better than Workstation in
this regard. Maybe we should put that in the FAQ?

I think it's safe to claim 2003 is better than XP, but I'm not sure
that's enough to generalize into server vs workstation yet. It
implies 2000 Server would be better than 2000 Pro, which might not be
true. I'm also wondering whether 64bit XP behaves differently, since
IIRC it's based on the 2003 kernel. Then there's Vista...

Unfortunately I don't have access to any of these versions to test
with at the moment.

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ***@postgresql.org so that your
message can get through to the mailing list cleanly

[Magnus Hagander]

Post by Trevor Talbot

Post by Magnus Hagander

Post by Trevor Talbot

Post by Magnus Hagander

Post by Trevor Talbot
As for desktop heap, only 65KB of the service heap was allocated, or
about 80 bytes per connection. No danger of hitting limits in the
kernel memory pools either.

As Dave said, it could be that the server version uses a lot less heap per
process, which would be another good reason to use server rather than XP to
run postgresql. But might there also be other differences, such as some
third party (or non-core microsoft) product installed?

The XP SP2 machine I tried 8.2.5 on was chewing up about 3.1KB per
process, and it's not running anything invasive (AV or otherwise).

Then I think we can claim that Server is just better than Workstation in
this regard. Maybe we should put that in the FAQ?

I think it's safe to claim 2003 is better than XP, but I'm not sure
that's enough to generalize into server vs workstation yet. It
implies 2000 Server would be better than 2000 Pro, which might not be
true. I'm also wondering whether 64bit XP behaves differently, since
IIRC it's based on the 2003 kernel. Then there's Vista...

Valid points, of course. Specifically, it'd be interesting to know where
Vista stands, and possibly 2008 server. I don't care that much about
2000, really.

I don't have installations of either one, though.. :-(

//Magnus

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

[Dave Page]

Post by Magnus Hagander
Taking this one to -hackers once and for all now...
Can you try the attached patch? See how many backends you can get up to.

Regression tests run just fine, and I've run multiple pgbench runs with
3 and 4 sessions of 100 connections each*, with pgAdmin monitoring
things at the same time. Saw up to 403 simultanteous connections in
pg_stat_activity, and the system remained stable and responsive, albeit
somewhat slower than normal.

So, 400 connections on a 2.33GHz MacBook Pro running XP Pro with 2GB RAM
- thats not too shabby :-)

/D


* For those that weren't peering over Magnus' or Greg's shoulder during
various IM discussions over the last few days, I've found that the ~125
connection ceiling I was hitting when running from a command prompt was
actually an as yet unsolved problem in pgbench, not the server. Multiple
pgbench sessions seem to run just fine if kept to around 100 connections
each.


---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

[Tom Lane]

Post by Magnus Hagander
I was planning to make it even easier and let Windows do the job for us,
just using RegisterWaitForSingleObject(). Does the same - one thread per
64 backends, but we don't have to deal with the queueing ourselves.
Should be rather trivial to do.

How can that possibly work? Backends have to be able to run
concurrently, and I don't see how they'll do that if they share a stack.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

[Magnus Hagander]

Post by Tom Lane

Post by Magnus Hagander
I was planning to make it even easier and let Windows do the job for us,
just using RegisterWaitForSingleObject(). Does the same - one thread per
64 backends, but we don't have to deal with the queueing ourselves.
Should be rather trivial to do.

How can that possibly work? Backends have to be able to run
concurrently, and I don't see how they'll do that if they share a stack.

We're not talking about the backends, we're talking about the backend
waiter threads whose sole purpose is to wait for a backend to die and
then raise a signal when it does. We can easily have the kernel wait for
a whole bunch of them at once, and have it call our callback function
whenever anyone of them dies.

//Magnus

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

[Tom Lane]

Post by Magnus Hagander
We're not talking about the backends, we're talking about the backend
waiter threads whose sole purpose is to wait for a backend to die and
then raise a signal when it does.

Oh, OK, I had not twigged to exactly what the threads were being used
for. Never mind ...

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

[Trevor Talbot]

Post by Tom Lane

Post by Magnus Hagander
I was planning to make it even easier and let Windows do the job for us,
just using RegisterWaitForSingleObject(). Does the same - one thread per
64 backends, but we don't have to deal with the queueing ourselves.
Should be rather trivial to do.

How can that possibly work? Backends have to be able to run
concurrently, and I don't see how they'll do that if they share a stack.

This is about what postmaster does for its SIGCHLD wait equivalent on
win32. The 64 comes from Windows' object/event mechanism, which lets
you perform a blocking wait on up to that many handles in a single
call. Currently postmaster is creating a new thread to wait on only
one backend at a time, so it ends up with too many threads.

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

[Magnus Hagander]

Post by Magnus Hagander
Oh, that's interesting. That's actually a sideeffect of us increasing
the stack size for the postgres.exe executable in order to work on other
things. By default, it burns 1MB/thread, but ours will do 4MB. Never
really thought of the problem that it'll run out of address space.
Unfortunately, that size can't be changed in the CreateThread() call -
only the initially committed size can be changed there.

Windows XP supports the STACK_SIZE_PARAM_IS_A_RESERVATION flag, which
apparently allows to reduce the reserved size. It might be better to do
this the other way round, though (leave the reservation at its 1 MB
default, and increase it only when necessary).

It does, but we still support windows 2000 as well. I think it's better
to use a different method altogether - one not using one thread per child.

//Magnus

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

[Florian Weimer]

Post by Magnus Hagander
Oh, that's interesting. That's actually a sideeffect of us increasing
the stack size for the postgres.exe executable in order to work on other
things. By default, it burns 1MB/thread, but ours will do 4MB. Never
really thought of the problem that it'll run out of address space.
Unfortunately, that size can't be changed in the CreateThread() call -
only the initially committed size can be changed there.

Windows XP supports the STACK_SIZE_PARAM_IS_A_RESERVATION flag, which
apparently allows to reduce the reserved size. It might be better to do
this the other way round, though (leave the reservation at its 1 MB
default, and increase it only when necessary).

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

[Magnus Hagander]

Post by Dave Page

Post by Magnus Hagander
Taking this one to -hackers once and for all now...
Can you try the attached patch? See how many backends you can get up to.

Regression tests run just fine, and I've run multiple pgbench runs with
3 and 4 sessions of 100 connections each*, with pgAdmin monitoring
things at the same time. Saw up to 403 simultanteous connections in
pg_stat_activity, and the system remained stable and responsive, albeit
somewhat slower than normal.

What was the memory space consumption of the postmaster process, and compared to without the patch?

VM size in taskmgr should show that I think, and should show a much smaller footprint now..

/Magnus


---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

[Dave Page]

Post by Magnus Hagander
VM size in taskmgr should show that I think, and should show a much smaller footprint now..

With patch - 4,492K
Without patch: 28,224K

Thats with 3 x 100 pgbench connections.

/D


---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

[Magnus Hagander]

Post by Dave Page

Post by Magnus Hagander
VM size in taskmgr should show that I think, and should show a much
smaller footprint now..

With patch - 4,492K
Without patch: 28,224K
Thats with 3 x 100 pgbench connections.

That's nice!

But. That can't be address space usage, it has to be actual memory
usage. Since each thread should chew up 4Mb of address space, and
there's at least two threads in there :-) So looking at the VM column
was obviously not correct.
* looks up some docs*
Right. You need to look at VM size in *process explorer*. VM size in
task manager has nothing to do with VM size, it's the private bytes :-S
And there is no way to see that info from task manager, I think. PE is
your friend.


Anyway. Other than a refresher on those, I'd be interested in two other
important parts:
* How many threads does it reach when you have 300 active backends?
* Is there a handle leak? meaning once your 300 backends have exited,
does the number of handles in the process drop down to the same value it
had before?

(sorry, wish I was in a position to run these tests myself, but I'm not
right now)

//Magnus


---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

[Dave Page]

Post by Magnus Hagander
Right. You need to look at VM size in *process explorer*. VM size in
task manager has nothing to do with VM size, it's the private bytes :-S
And there is no way to see that info from task manager, I think. PE is
your friend.
Anyway. Other than a refresher on those, I'd be interested in two other
* How many threads does it reach when you have 300 active backends?
* Is there a handle leak? meaning once your 300 backends have exited,
does the number of handles in the process drop down to the same value it
had before?

Without patch:

VM: 1,322,792K
Idle threads: 6
Peak threads: 306
Handles at start: 576
Handles at end: 576

With patch:

VM: 98,088K
Idle threads: 3
Peak threads: 7
Handles at start: 576
Handles at end: 585 (585 again after second run).

/D


---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

[Magnus Hagander]

Post by Dave Page

Post by Magnus Hagander
Right. You need to look at VM size in *process explorer*. VM size in
task manager has nothing to do with VM size, it's the private bytes :-S
And there is no way to see that info from task manager, I think. PE is
your friend.
Anyway. Other than a refresher on those, I'd be interested in two other
* How many threads does it reach when you have 300 active backends?
* Is there a handle leak? meaning once your 300 backends have exited,
does the number of handles in the process drop down to the same value it
had before?

VM: 1,322,792K
Idle threads: 6
Peak threads: 306
Handles at start: 576
Handles at end: 576
VM: 98,088K
Idle threads: 3
Peak threads: 7
Handles at start: 576
Handles at end: 585 (585 again after second run).

Ah, now we're talking. That's the kind of reduction I was looking for :-)

I think the difference in handles is because the threadpool keeps some
things around. As long as it stays at 585 and comes back down after a
second run, we're fine at that - there's no leak.

Attached is an updated version of the patch, currently being tested by
both me and Dave. If it passes our tests, I'll apply this so it gets
included for broader testing in beta2.


//Magnus

[Magnus Hagander]

Post by Magnus Hagander

Post by Dave Page

Post by Magnus Hagander
Right. You need to look at VM size in *process explorer*. VM size in
task manager has nothing to do with VM size, it's the private bytes :-S
And there is no way to see that info from task manager, I think. PE is
your friend.
Anyway. Other than a refresher on those, I'd be interested in two other
* How many threads does it reach when you have 300 active backends?
* Is there a handle leak? meaning once your 300 backends have exited,
does the number of handles in the process drop down to the same value it
had before?

VM: 1,322,792K
Idle threads: 6
Peak threads: 306
Handles at start: 576
Handles at end: 576
VM: 98,088K
Idle threads: 3
Peak threads: 7
Handles at start: 576
Handles at end: 585 (585 again after second run).

Ah, now we're talking. That's the kind of reduction I was looking for :-)
I think the difference in handles is because the threadpool keeps some
things around. As long as it stays at 585 and comes back down after a
second run, we're fine at that - there's no leak.
Attached is an updated version of the patch, currently being tested by
both me and Dave. If it passes our tests, I'll apply this so it gets
included for broader testing in beta2.

So of course, for all good patches, problems turn up :-(

This patch doesn't work on mingw, at least not on all versions. The
reason is that, as usual, the mingw libraries are not complete. We've
dealt with this before, by dynamically loading the functions. We know
this works. But I don't have time to fix that tonight, and I'll be
offline much of this weekend.

Now, given these great improvements, I'd very much like this in beta2,
so it can get proper testing. This leaves us with a couple of choices:

1) Delay beta2 until the beginning of next week. I'll get this fixed
sunday evening or monday at the latest. I know how to fix it, it's just
that I don't have the time right now :( (This assumes that the plan
still is to wrap beta2 today)

2) Apply the patch as-is. Then beta2 will work fine with the msvc build,
which is used for the binary distribution. But it's broken on mingw
until fixed, which of course includes the buildfarm stuff. Again, we
know how to fix this.

2b) I apply this as-is, and someone else cleans up mingw before beta2 is
wrapped.

3) We don't apply this, and wait until beta3 to have it tested.
Depending on how many betas we end up with, that may leave us with very
little testing before release.



2b is of course the best here, but then someone has to step up and
volunteer to do that.

I'm leaning towards applying the patch now, and hoping for 2b to happen.
If it doesn't happen, the choice between 1 and 2 can be made when the
time to wrap the beta approaches (at which point I will be offline, so I
escape :-P)


The patch that would go in is the one previously posted plus a couple of
minor edits in comments as suggested on IM by Alvaro.


Comments?

//Magnus

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

[Dave Page]

Post by Magnus Hagander
I'm leaning towards applying the patch now, and hoping for 2b to happen.

I think we should live with the mingw BF breakage for a day or two. The
patch is clearly an important improvement, but it should be as widely
tested as possible.

/D

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

[Joshua D. Drake]

On Fri, 26 Oct 2007 21:58:03 +0100

Post by Dave Page

Post by Magnus Hagander
I'm leaning towards applying the patch now, and hoping for 2b to happen.

I think we should live with the mingw BF breakage for a day or two.
The patch is clearly an important improvement, but it should be as
widely tested as possible.

I think this makes sense. I certainly don't want to hold back Beta2 and
this patch so far is an obvious improvement.

Sincerely,

Joshua D. Drake

Post by Dave Page
/D
---------------------------(end of
broadcast)--------------------------- TIP 2: Don't 'kill -9' the
postmaster

[Magnus Hagander]

Post by Joshua D. Drake
On Fri, 26 Oct 2007 21:58:03 +0100

Post by Dave Page

Post by Magnus Hagander
I'm leaning towards applying the patch now, and hoping for 2b to happen.

I think we should live with the mingw BF breakage for a day or two.
The patch is clearly an important improvement, but it should be as
widely tested as possible.

I think this makes sense. I certainly don't want to hold back Beta2 and
this patch so far is an obvious improvement.

I read the consensus of this thread as "apply the patch as-is and let's
fix mingw as soon as we can", so this is what I've done. If I got it
wrong, feel free to back out :-)

//Magnus

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

[Tom Lane]

Post by Dave Page

Post by Magnus Hagander
I'm leaning towards applying the patch now, and hoping for 2b to happen.

I think we should live with the mingw BF breakage for a day or two. The
patch is clearly an important improvement, but it should be as widely
tested as possible.

If Dave is intending to build the Windows installer as soon as beta2 is
wrapped, then I agree with this plan. But my understanding was we'd
missed his window for today and so that wouldn't happen till Monday.
Assuming that's true, the idea Bruce and I had discussed on the phone
was:

1. Wrap official beta2 tonight, so that other packagers can work from it
over the weekend;

2. Magnus fixes his patch Sunday and applies it then;

3. Dave builds the Windows installer Monday *from CVS tip*.

So the Windows version would be beta2-plus-a-little but we'd neither
hold up work on other platforms nor break anything in buildfarm.

Just an alternative thought. In any case I agree that we want
Windows testing of beta2 to include this patch.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ***@postgresql.org so that your
message can get through to the mailing list cleanly

[Magnus Hagander]

Post by Tom Lane

Post by Dave Page

Post by Magnus Hagander
I'm leaning towards applying the patch now, and hoping for 2b to happen.

I think we should live with the mingw BF breakage for a day or two. The
patch is clearly an important improvement, but it should be as widely
tested as possible.

If Dave is intending to build the Windows installer as soon as beta2 is
wrapped, then I agree with this plan. But my understanding was we'd
missed his window for today and so that wouldn't happen till Monday.
Assuming that's true, the idea Bruce and I had discussed on the phone
1. Wrap official beta2 tonight, so that other packagers can work from it
over the weekend;
2. Magnus fixes his patch Sunday and applies it then;
3. Dave builds the Windows installer Monday *from CVS tip*.
So the Windows version would be beta2-plus-a-little but we'd neither
hold up work on other platforms nor break anything in buildfarm.
Just an alternative thought. In any case I agree that we want
Windows testing of beta2 to include this patch.

If we do that, then what we label as beta2 in the installer is *not* the
same as someone who has built beta2 from source. Can't say I like that
one, and I know Dave doesn't like it (he said so before going offline).

I'd rather see msvc working and mingw broken for beta2 really, because
then we *know* that if someone says they're doing beta2 on mingw they're
misinformed...

//Magnus

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ***@postgresql.org so that your
message can get through to the mailing list cleanly

[Magnus Hagander]

Post by Magnus Hagander
Attached is an updated version of the patch, currently being tested by
both me and Dave. If it passes our tests, I'll apply this so it gets
included for broader testing in beta2.

One question: what's this about?

Post by Magnus Hagander
+ #define _WIN32_WINNT 0x0500

This looks like it might be tightening our assumptions about which
Windows flavors we can run on. I'm not necessarily against that,
but it should be publicly discussed if it's happening.

It enables Windows 2000-specific headers. We already require Windows
2000 to run, so it doesn't restrict us anymore than we already are. It
just exposes those parts of the header files.

//Magnus

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

[Tom Lane]

Post by Magnus Hagander
Attached is an updated version of the patch, currently being tested by
both me and Dave. If it passes our tests, I'll apply this so it gets
included for broader testing in beta2.

One question: what's this about?

Post by Magnus Hagander
+ #define _WIN32_WINNT 0x0500

This looks like it might be tightening our assumptions about which
Windows flavors we can run on. I'm not necessarily against that,
but it should be publicly discussed if it's happening.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

[Dave Page]

------- Original Message -------
Sent: 10/11/07, 23:17:13
Subject: Re: [HACKERS] 8.2.3: Server crashes on Windows using Eclipse/Junit
As for desktop heap, only 65KB of the service heap was allocated, or
about 80 bytes per connection. No danger of hitting limits in the
kernel memory pools either.

That's interesting (and useful to know) - server is clearly not using nearly as much desktop heap when initialising user32 as XP.

Available RAM seems like a pretty reasonable limit to me ;)

Yup.

/D

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq