Discussion:
pg_receivexlog --status-interval add fsync feedback
f***@pm.nttdata.co.jp
2014-08-12 09:19:58 UTC
Permalink
Hi all,

This patch is to add setting to send status packets after fsync to --status-interval of pg_receivexlog.

If -1 is specified to --status-interval, status packets is sent as soon as after fsync.

Others are the same as when 0 is specified to --status-interval.
When requested by the server, send the status packet.

To use replication slot, and specify -1 to --fsync-interval.
As a result, simple WAL synchronous replication can be performed.

In simple WAL synchronization, it is possible to test the replication.
If there was F/O in synchronous replication,
it is available in the substitute until standby is restored.

Regards,

--
Furuya Osamu
Fujii Masao
2014-08-12 10:13:24 UTC
Permalink
On Tue, Aug 12, 2014 at 6:19 PM, <***@pm.nttdata.co.jp> wrote:
> Hi all,
>
> This patch is to add setting to send status packets after fsync to --status-interval of pg_receivexlog.
>
> If -1 is specified to --status-interval, status packets is sent as soon as after fsync.

I don't think that it's good idea to control that behavior by using
--status-interval. I'm sure that there are some users who both want
that behavior and want set the maximum interval between a feedback
is sent back to the server because these settings are available in
walreceiver. But your version of --status-interval doesn't allow those
settings at all. That is, if set to -1, the maximum interval between
each feedback cannot be set. OTOH, if set to the positive number,
feedback-as-soon-as-fsync behavior cannot be enabled. Therefore,
I'm thinking that it's better to introduce new separate option for that
behavior.

Regards,

--
Fujii Masao


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
f***@pm.nttdata.co.jp
2014-08-13 08:55:23 UTC
Permalink
> I don't think that it's good idea to control that behavior by using
> --status-interval. I'm sure that there are some users who both want that
> behavior and want set the maximum interval between a feedback is sent
> back to the server because these settings are available in walreceiver.
> But your version of --status-interval doesn't allow those settings at
> all. That is, if set to -1, the maximum interval between each feedback
> cannot be set. OTOH, if set to the positive number,
> feedback-as-soon-as-fsync behavior cannot be enabled. Therefore, I'm
> thinking that it's better to introduce new separate option for that
> behavior.

Thanks for the review!
This patch was split option as you pointed out.

If -r option is specified, status packets sent to server as soon as after fsync.
Otherwise to send server status packet in the spacing of the --status-interval.

Regards,

--
Furuya Osamu
Sawada Masahiko
2014-08-16 06:48:17 UTC
Permalink
On Wed, Aug 13, 2014 at 5:55 PM, <***@pm.nttdata.co.jp> wrote:
>> I don't think that it's good idea to control that behavior by using
>> --status-interval. I'm sure that there are some users who both want that
>> behavior and want set the maximum interval between a feedback is sent
>> back to the server because these settings are available in walreceiver.
>> But your version of --status-interval doesn't allow those settings at
>> all. That is, if set to -1, the maximum interval between each feedback
>> cannot be set. OTOH, if set to the positive number,
>> feedback-as-soon-as-fsync behavior cannot be enabled. Therefore, I'm
>> thinking that it's better to introduce new separate option for that
>> behavior.
>
> Thanks for the review!
> This patch was split option as you pointed out.
>
> If -r option is specified, status packets sent to server as soon as after fsync.
> Otherwise to send server status packet in the spacing of the --status-interval.
>

Hi,

I took a look at this patch.
I applied patch to master successfully, and did not get error with compiling.
Also it works fine.

One question is why reply_fsync is defined as volatile variable?
Sorry I could not understand reason of that.

Currently patch modifies argument of some function (e.g., Handle
CopyStream, Process LogDate Msg), and add the similar code to each
function.
I don't think it is good approach.
For example, I think that we should gather these code into one function.

Thought?

Regards,

-------
Sawada Masahiko


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
f***@pm.nttdata.co.jp
2014-08-18 10:55:36 UTC
Permalink
Thanks for the review!

> One question is why reply_fsync is defined as volatile variable?
> Sorry I could not understand reason of that.

It was affected to time_to_abort -- since it is unnecessary, it deletes.

> Currently patch modifies argument of some function (e.g., Handle
> CopyStream, Process LogDate Msg), and add the similar code to each
> function.
> I don't think it is good approach.
> For example, I think that we should gather these code into one function.

Feedback was judged immediately after each fsync until now.
I revised it in reference to walreceiver.
Feedback of fsync is judged together with the judgment of --status-interval.
Thereby, the specification to an argument became minimum.

Regards,

--
Furuya Osamu
Sawada Masahiko
2014-08-18 14:45:10 UTC
Permalink
On Mon, Aug 18, 2014 at 7:55 PM, <***@pm.nttdata.co.jp> wrote:
> Thanks for the review!
>
>> One question is why reply_fsync is defined as volatile variable?
>> Sorry I could not understand reason of that.
>
> It was affected to time_to_abort -- since it is unnecessary, it deletes.
>
>> Currently patch modifies argument of some function (e.g., Handle
>> CopyStream, Process LogDate Msg), and add the similar code to each
>> function.
>> I don't think it is good approach.
>> For example, I think that we should gather these code into one function.
>
> Feedback was judged immediately after each fsync until now.
> I revised it in reference to walreceiver.
> Feedback of fsync is judged together with the judgment of --status-interval.
> Thereby, the specification to an argument became minimum.

Thank you for updating the patch.

I did not get error with applying, and compiling.
It works fine. I think this function code has no problem.
Could you please submit patch to commit fest app?

Regards,

-------
Sawada Masahiko


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
f***@pm.nttdata.co.jp
2014-08-19 00:52:39 UTC
Permalink
> Thank you for updating the patch.
>
> I did not get error with applying, and compiling.
> It works fine. I think this function code has no problem.
> Could you please submit patch to commit fest app?

Thanks for the review!

As you pointed out, submitted patch to commit fest app.

Regards,

--
Furuya Osamu

--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.post
Fujii Masao
2014-08-19 12:19:33 UTC
Permalink
On Tue, Aug 19, 2014 at 9:52 AM, <***@pm.nttdata.co.jp> wrote:
>> Thank you for updating the patch.
>>
>> I did not get error with applying, and compiling.
>> It works fine. I think this function code has no problem.
>> Could you please submit patch to commit fest app?
>
> Thanks for the review!
>
> As you pointed out, submitted patch to commit fest app.

When replication slot is not specified in pg_receivexlog, the flush location
in the feedback message always indicates invalid. So there seems to be
no need to send the feedback as soon as fsync is issued, in that case.
How should this option work when replication slot is not specified?

Regards,

--
Fujii Masao


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
f***@pm.nttdata.co.jp
2014-08-21 05:54:17 UTC
Permalink
> When replication slot is not specified in pg_receivexlog, the flush
> location in the feedback message always indicates invalid. So there seems
> to be no need to send the feedback as soon as fsync is issued, in that
> case.
> How should this option work when replication slot is not specified?

Thanks for the review!

The present is not checking the existence of specification of -S.

The use case when replication slot is not specified.

Because it does fsync, it isn't an original intention.
remote_write is set in synchronous_commit.

To call attention to the user, append following documents.
"If you want to report the flush position to the server, should use -S option."

Regards,

--
Furuya Osamu
Sawada Masahiko
2014-08-21 09:19:39 UTC
Permalink
On Thu, Aug 21, 2014 at 2:54 PM, <***@pm.nttdata.co.jp> wrote:
>> When replication slot is not specified in pg_receivexlog, the flush
>> location in the feedback message always indicates invalid. So there seems
>> to be no need to send the feedback as soon as fsync is issued, in that
>> case.
>> How should this option work when replication slot is not specified?
>
> Thanks for the review!
>
> The present is not checking the existence of specification of -S.
>
> The use case when replication slot is not specified.
>
> Because it does fsync, it isn't an original intention.
> remote_write is set in synchronous_commit.
>
> To call attention to the user, append following documents.
> "If you want to report the flush position to the server, should use -S option."
>

Thank you for updating the patch.
I reviewed the patch.

First of all, I think that we should not append the above message to
section of '-r' option.
(Or these message might not be needed at all)
Whether flush location in feedback message is valid, is not depend on
'-r' option.

If we use '-r' option and 'S' option (i.g., replication slot) then
pg_receivexlog informs valid flush
location to primary server at the same time as doing fsync.
But, if we don't specify replication slot then the flush location in
feedback message always invalid.
So I think Fujii-san pointed out that sending of invalid flush
location is not needed
if pg_receivexlog does not use replication slot.

Regards,

-------
Sawada Masahiko


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
f***@pm.nttdata.co.jp
2014-08-22 04:35:48 UTC
Permalink
> Thank you for updating the patch.
> I reviewed the patch.
>
> First of all, I think that we should not append the above message to
> section of '-r' option.
> (Or these message might not be needed at all) Whether flush location in
> feedback message is valid, is not depend on '-r' option.
>
> If we use '-r' option and 'S' option (i.g., replication slot) then
> pg_receivexlog informs valid flush location to primary server at the same
> time as doing fsync.
> But, if we don't specify replication slot then the flush location in
> feedback message always invalid.
> So I think Fujii-san pointed out that sending of invalid flush location
> is not needed if pg_receivexlog does not use replication slot.

Thanks for the review!

I understand the attention message wasn't appropriate.

To report the write location, even If you do not specify a replication slot.
So the fix only appended messages.

There was a description of the flush location section of '-S' option,
but I intended to catch eye more and added a message.

Is it better to make specification of the -S option indispensable?

Regards,

--
Furuya Osamu

--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgs
Fujii Masao
2014-09-05 03:10:32 UTC
Permalink
On Fri, Aug 22, 2014 at 1:35 PM, <***@pm.nttdata.co.jp> wrote:
>> Thank you for updating the patch.
>> I reviewed the patch.
>>
>> First of all, I think that we should not append the above message to
>> section of '-r' option.
>> (Or these message might not be needed at all) Whether flush location in
>> feedback message is valid, is not depend on '-r' option.
>>
>> If we use '-r' option and 'S' option (i.g., replication slot) then
>> pg_receivexlog informs valid flush location to primary server at the same
>> time as doing fsync.
>> But, if we don't specify replication slot then the flush location in
>> feedback message always invalid.
>> So I think Fujii-san pointed out that sending of invalid flush location
>> is not needed if pg_receivexlog does not use replication slot.
>
> Thanks for the review!
>
> I understand the attention message wasn't appropriate.
>
> To report the write location, even If you do not specify a replication slot.
> So the fix only appended messages.
>
> There was a description of the flush location section of '-S' option,
> but I intended to catch eye more and added a message.
>
> Is it better to make specification of the -S option indispensable?

The patch cannot be applied to HEAD cleanly. Could you update the patch?

Regards,

--
Fujii Masao


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
f***@pm.nttdata.co.jp
2014-09-05 05:51:27 UTC
Permalink
> > Thanks for the review!
> >
> > I understand the attention message wasn't appropriate.
> >
> > To report the write location, even If you do not specify a replication
> slot.
> > So the fix only appended messages.
> >
> > There was a description of the flush location section of '-S' option,
> > but I intended to catch eye more and added a message.
> >
> > Is it better to make specification of the -S option indispensable?
>
> The patch cannot be applied to HEAD cleanly. Could you update the patch?

Thank you for pointing out.
Updated the patch.

Regards,

--
Furuya Osamu
Heikki Linnakangas
2014-09-27 07:53:21 UTC
Permalink
On 09/05/2014 08:51 AM, ***@pm.nttdata.co.jp wrote:
>>> Thanks for the review!
>>>
>>> I understand the attention message wasn't appropriate.
>>>
>>> To report the write location, even If you do not specify a replication
>> slot.
>>> So the fix only appended messages.
>>>
>>> There was a description of the flush location section of '-S' option,
>>> but I intended to catch eye more and added a message.
>>>
>>> Is it better to make specification of the -S option indispensable?
>>
>> The patch cannot be applied to HEAD cleanly. Could you update the patch?
>
> Thank you for pointing out.
> Updated the patch.

I don't understand what this patch does. When would you want to use the
new --reply-fsync option? Is there any reason *not* to use it? In other
words, do we need an option for this, couldn't you just always send the
feedback message after fsync?

- Heikki


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
f***@pm.nttdata.co.jp
2014-09-29 10:13:59 UTC
Permalink
> On 09/05/2014 08:51 AM, ***@pm.nttdata.co.jp wrote:
> >>> Thanks for the review!
> >>>
> >>> I understand the attention message wasn't appropriate.
> >>>
> >>> To report the write location, even If you do not specify a
> >>> replication
> >> slot.
> >>> So the fix only appended messages.
> >>>
> >>> There was a description of the flush location section of '-S'
> >>> option, but I intended to catch eye more and added a message.
> >>>
> >>> Is it better to make specification of the -S option indispensable?
> >>
> >> The patch cannot be applied to HEAD cleanly. Could you update the
> patch?
> >
> > Thank you for pointing out.
> > Updated the patch.
>
> I don't understand what this patch does. When would you want to use the
> new --reply-fsync option? Is there any reason *not* to use it? In other
> words, do we need an option for this, couldn't you just always send the
> feedback message after fsync?

Thanks for the comment.

--reply-fsync option is intended for use in synchronous mode.

By specifying -F option and --slot option, process calls fsync() when it received the WAL, and flush location would be set in feedback message.

Interval of sending feedback message depends on -s option in this state, so in the case of synchronous mode, waiting for feedback message would occure.

therefore, --reply-fsync option is necessary. because it can send the feedback message after fsync without waiting for the interval of -s option.

The reason for not sending the feedback message after fsync without waiting for the interval of -s option always, is to answer the needs who want to use fsync only (NOT using synchronous mode).

Regards,

--
Furuya Osamu


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgr
Heikki Linnakangas
2014-10-06 12:06:53 UTC
Permalink
On 09/29/2014 01:13 PM, ***@pm.nttdata.co.jp wrote:
>> I don't understand what this patch does. When would you want to use
>> the new --reply-fsync option? Is there any reason *not* to use it?
>> In other words, do we need an option for this, couldn't you just
>> always send the feedback message after fsync?
>
> Thanks for the comment.
>
> --reply-fsync option is intended for use in synchronous mode.
>
> By specifying -F option and --slot option, process calls fsync() when
> it received the WAL, and flush location would be set in feedback
> message.
>
> Interval of sending feedback message depends on -s option in this
> state, so in the case of synchronous mode, waiting for feedback
> message would occure.
>
> therefore, --reply-fsync option is necessary. because it can send the
> feedback message after fsync without waiting for the interval of -s
> option.
>
> The reason for not sending the feedback message after fsync without
> waiting for the interval of -s option always, is to answer the needs
> who want to use fsync only (NOT using synchronous mode).

I still don't get it. AFAICS there are two ways to use pg_receivexlog.
Either you use it as a synchronous standby, or not.

What set of options would you pass if you want to use it as a
synchronous standby? And if you don't? Could we just have a single
"--synchronous" flag, instead of -F and --reply-fsync?

- Heikki


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
f***@pm.nttdata.co.jp
2014-10-08 04:23:14 UTC
Permalink
> On 09/29/2014 01:13 PM, ***@pm.nttdata.co.jp wrote:
> >> I don't understand what this patch does. When would you want to use
> >> the new --reply-fsync option? Is there any reason *not* to use it?
> >> In other words, do we need an option for this, couldn't you just
> >> always send the feedback message after fsync?
> >
> > Thanks for the comment.
> >
> > --reply-fsync option is intended for use in synchronous mode.
> >
> > By specifying -F option and --slot option, process calls fsync() when
> > it received the WAL, and flush location would be set in feedback
> > message.
> >
> > Interval of sending feedback message depends on -s option in this
> > state, so in the case of synchronous mode, waiting for feedback
> > message would occure.
> >
> > therefore, --reply-fsync option is necessary. because it can send the
> > feedback message after fsync without waiting for the interval of -s
> > option.
> >
> > The reason for not sending the feedback message after fsync without
> > waiting for the interval of -s option always, is to answer the needs
> > who want to use fsync only (NOT using synchronous mode).
>
> I still don't get it. AFAICS there are two ways to use pg_receivexlog.
> Either you use it as a synchronous standby, or not.
>
> What set of options would you pass if you want to use it as a synchronous
> standby? And if you don't? Could we just have a single "--synchronous"
> flag, instead of -F and --reply-fsync?

Thanks for comment.

If you set "synchronous_commit" as "remote_write", the options would be different .
The set of options in each case, see the following.


Synchronous standby(synchronous_commit=on)
--fsync-interval=-1
--reply-fsync
--slot=slotname

Synchronous standby(synchronous_commit=remote_write)
--fsync-interval=-1
--reply-fsync

Asynchronous
There are no relative options.


Well, if the response time delay(value of "--status-interval=interval") is acceptable, "--reply-fsync" is unnecessary.
Instead of "--reply-fsync", using "--synchronous"(which summarizes the "--reply-fsync" and "fsync-interval = -1") might be easy to understand. Although, in that case, "--fsync-interval=interval" would be fixed value. Isn't there any problem ?

Regards,

--
Furuya Osamu


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.
Heikki Linnakangas
2014-10-08 06:59:59 UTC
Permalink
On 10/08/2014 07:23 AM, ***@pm.nttdata.co.jp wrote:
>> What set of options would you pass if you want to use it as a synchronous
>> standby? And if you don't? Could we just have a single "--synchronous"
>> flag, instead of -F and --reply-fsync?
>
> If you set "synchronous_commit" as "remote_write", the options would be different .
> The set of options in each case, see the following.
>
>
> Synchronous standby(synchronous_commit=on)
> --fsync-interval=-1
> --reply-fsync
> --slot=slotname
>
> Synchronous standby(synchronous_commit=remote_write)
> --fsync-interval=-1
> --reply-fsync
>
> Asynchronous
> There are no relative options.
>
>
> Well, if the response time delay(value of "--status-interval=interval") is acceptable, "--reply-fsync" is unnecessary.
> Instead of "--reply-fsync", using "--synchronous"(which summarizes the "--reply-fsync" and "fsync-interval = -1") might be easy to understand. Although, in that case, "--fsync-interval=interval" would be fixed value. Isn't there any problem ?

I think we should remove --fsync-interval and --reply-fsync, and just
have a --synchronous option, which would imply the same behavior you get
with --fsync-interval=-1 --reply--fsync.

That leaves the question of whether pg_receivexlog should do fsyncs when
it's not acting as a synchronous standby. There isn't any real need to
do so. In asynchronous mode, there are no guarantees anyway, and even
without an fsync, the OS will eventually flush the data to disk.

But we could do even better than that. It would be best if you didn't
need even the --synchronous flag. The server knows whether the client is
a synchronous standby or not. It could tell the client.

- Heikki



--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
f***@pm.nttdata.co.jp
2014-10-08 08:47:16 UTC
Permalink
> On 10/08/2014 07:23 AM, ***@pm.nttdata.co.jp wrote:
> >> What set of options would you pass if you want to use it as a
> >> synchronous standby? And if you don't? Could we just have a single
> "--synchronous"
> >> flag, instead of -F and --reply-fsync?
> >
> > If you set "synchronous_commit" as "remote_write", the options would
> be different .
> > The set of options in each case, see the following.
> >
> >
> > Synchronous standby(synchronous_commit=on)
> > --fsync-interval=-1
> > --reply-fsync
> > --slot=slotname
> >
> > Synchronous standby(synchronous_commit=remote_write)
> > --fsync-interval=-1
> > --reply-fsync
> >
> > Asynchronous
> > There are no relative options.
> >
> >
> > Well, if the response time delay(value of
> "--status-interval=interval") is acceptable, "--reply-fsync" is
> unnecessary.
> > Instead of "--reply-fsync", using "--synchronous"(which summarizes
> the "--reply-fsync" and "fsync-interval = -1") might be easy to
> understand. Although, in that case, "--fsync-interval=interval" would
> be fixed value. Isn't there any problem ?
>
> I think we should remove --fsync-interval and --reply-fsync, and just
> have a --synchronous option, which would imply the same behavior you get
> with --fsync-interval=-1 --reply--fsync.
>
> That leaves the question of whether pg_receivexlog should do fsyncs when
> it's not acting as a synchronous standby. There isn't any real need to
> do so. In asynchronous mode, there are no guarantees anyway, and even
> without an fsync, the OS will eventually flush the data to disk.
>
> But we could do even better than that. It would be best if you didn't
> need even the --synchronous flag. The server knows whether the client
> is a synchronous standby or not. It could tell the client.

Thanks for the reply.

If we remove --fsync-interval, resoponse time to user will not be delay.
Although, fsync will be executed multiple times in a short period.
And there is no way to solve the problem without --fsync-interval, what should we do about it?

In asynchronous mode, I think there is no problem since the specification is same with release version.

> The server knows whether the client is a synchronous standby or not.
> It could tell the client.

When notifying the synchronous/asynchronous mode to the client from the server,
do we need to change the format of the Message ?

Regards,

--
Furuya Osamu

--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpr
Heikki Linnakangas
2014-10-08 11:50:07 UTC
Permalink
On 10/08/2014 11:47 AM, ***@pm.nttdata.co.jp wrote:
>> On 10/08/2014 07:23 AM, ***@pm.nttdata.co.jp wrote:
>>>> What set of options would you pass if you want to use it as a
>>>> synchronous standby? And if you don't? Could we just have a single
>> "--synchronous"
>>>> flag, instead of -F and --reply-fsync?
>>>
>>> If you set "synchronous_commit" as "remote_write", the options would
>> be different .
>>> The set of options in each case, see the following.
>>>
>>>
>>> Synchronous standby(synchronous_commit=on)
>>> --fsync-interval=-1
>>> --reply-fsync
>>> --slot=slotname
>>>
>>> Synchronous standby(synchronous_commit=remote_write)
>>> --fsync-interval=-1
>>> --reply-fsync
>>>
>>> Asynchronous
>>> There are no relative options.
>>>
>>>
>>> Well, if the response time delay(value of
>> "--status-interval=interval") is acceptable, "--reply-fsync" is
>> unnecessary.
>>> Instead of "--reply-fsync", using "--synchronous"(which summarizes
>> the "--reply-fsync" and "fsync-interval = -1") might be easy to
>> understand. Although, in that case, "--fsync-interval=interval" would
>> be fixed value. Isn't there any problem ?
>>
>> I think we should remove --fsync-interval and --reply-fsync, and just
>> have a --synchronous option, which would imply the same behavior you get
>> with --fsync-interval=-1 --reply--fsync.
>>
>> That leaves the question of whether pg_receivexlog should do fsyncs when
>> it's not acting as a synchronous standby. There isn't any real need to
>> do so. In asynchronous mode, there are no guarantees anyway, and even
>> without an fsync, the OS will eventually flush the data to disk.
>>
>> But we could do even better than that. It would be best if you didn't
>> need even the --synchronous flag. The server knows whether the client
>> is a synchronous standby or not. It could tell the client.
>
> If we remove --fsync-interval, resoponse time to user will not be delay.
> Although, fsync will be executed multiple times in a short period.
> And there is no way to solve the problem without --fsync-interval, what should we do about it?

I'm sorry, I didn't understand that.

> In asynchronous mode, I think there is no problem since the specification is same with release version.
>
>> The server knows whether the client is a synchronous standby or not.
>> It could tell the client.
>
> When notifying the synchronous/asynchronous mode to the client from the server,
> do we need to change the format of the Message ?

Yeah. Or rather, add a new message type, to indicate the
synchronous/asynchronous status.

- Heikki



--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
f***@pm.nttdata.co.jp
2014-10-09 04:47:11 UTC
Permalink
> >>>> What set of options would you pass if you want to use it as a
> >>>> synchronous standby? And if you don't? Could we just have a single
> >> "--synchronous"
> >>>> flag, instead of -F and --reply-fsync?
> >>>
> >>> If you set "synchronous_commit" as "remote_write", the options would
> >> be different .
> >>> The set of options in each case, see the following.
> >>>
> >>>
> >>> Synchronous standby(synchronous_commit=on)
> >>> --fsync-interval=-1
> >>> --reply-fsync
> >>> --slot=slotname
> >>>
> >>> Synchronous standby(synchronous_commit=remote_write)
> >>> --fsync-interval=-1
> >>> --reply-fsync
> >>>
> >>> Asynchronous
> >>> There are no relative options.
> >>>
> >>>
> >>> Well, if the response time delay(value of
> >> "--status-interval=interval") is acceptable, "--reply-fsync" is
> >> unnecessary.
> >>> Instead of "--reply-fsync", using "--synchronous"(which summarizes
> >> the "--reply-fsync" and "fsync-interval = -1") might be easy to
> >> understand. Although, in that case, "--fsync-interval=interval"
> >> would be fixed value. Isn't there any problem ?
> >>
> >> I think we should remove --fsync-interval and --reply-fsync, and just
> >> have a --synchronous option, which would imply the same behavior you
> >> get with --fsync-interval=-1 --reply--fsync.
> >>
> >> That leaves the question of whether pg_receivexlog should do fsyncs
> >> when it's not acting as a synchronous standby. There isn't any real
> >> need to do so. In asynchronous mode, there are no guarantees anyway,
> >> and even without an fsync, the OS will eventually flush the data to
> disk.
> >>
> >> But we could do even better than that. It would be best if you didn't
> >> need even the --synchronous flag. The server knows whether the client
> >> is a synchronous standby or not. It could tell the client.
> >
> > If we remove --fsync-interval, resoponse time to user will not be delay.
> > Although, fsync will be executed multiple times in a short period.
> > And there is no way to solve the problem without --fsync-interval, what
> should we do about it?
>
> I'm sorry, I didn't understand that.
>
> > In asynchronous mode, I think there is no problem since the
> specification is same with release version.
> >
> >> The server knows whether the client is a synchronous standby or not.
> >> It could tell the client.
> >
> > When notifying the synchronous/asynchronous mode to the client from
> > the server, do we need to change the format of the Message ?
>
> Yeah. Or rather, add a new message type, to indicate the
> synchronous/asynchronous status.

Thanks for the reply.

> > If we remove --fsync-interval, resoponse time to user will not be delay.
> > Although, fsync will be executed multiple times in a short period.
> > And there is no way to solve the problem without --fsync-interval, what
> should we do about it?
>
> I'm sorry, I didn't understand that.

Here is an example.
When WAL is sent at 100ms intervals, fsync() is executed 10 times per second.
If --fsync-interval is set by 1 second, we have to wait SQL responce(kind of making WAL record) for 1 second, though, fsync() won't be executed several times in 1 second.
I think --fsync-interval meets the demands of people who wants to restrain fsync() happens for several time in short period, but what do you think?
Is it ok to delete --fsync-interval ?

> >> The server knows whether the client is a synchronous standby or not.
> >> It could tell the client.
> >
> > When notifying the synchronous/asynchronous mode to the client from
> > the server, do we need to change the format of the Message ?
>
> Yeah. Or rather, add a new message type, to indicate the
> synchronous/asynchronous status.

What kind of 'message type' we have to add ?

Do we need to separate 'w' into two types ? synchronous and asynchronous ?

OR

Add a new message type, kind of 'notify synchronous',
and inform pg_receivexlog of synchronous status when it connect to the server.

Regards,

--
Furuya Osamu

--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgre
Heikki Linnakangas
2014-10-09 06:37:39 UTC
Permalink
On 10/09/2014 07:47 AM, ***@pm.nttdata.co.jp wrote:
>>> If we remove --fsync-interval, resoponse time to user will not be delay.
>>> Although, fsync will be executed multiple times in a short period.
>>> And there is no way to solve the problem without --fsync-interval, what
>> should we do about it?
>>
>> I'm sorry, I didn't understand that.
>
> Here is an example.
> When WAL is sent at 100ms intervals, fsync() is executed 10 times per second.
> If --fsync-interval is set by 1 second, we have to wait SQL responce(kind of making WAL record) for 1 second, though, fsync() won't be executed several times in 1 second.
> I think --fsync-interval meets the demands of people who wants to restrain fsync() happens for several time in short period, but what do you think?
> Is it ok to delete --fsync-interval ?

I still don't see the problem.

In synchronous mode, pg_receivexlog should have similar logic as
walreceiver does. It should read as much WAL from the socket as it can
without blocking, and fsync() and send reply after that. And also fsync
whenever switching to new segment. If the master sends WAL every 100ms,
then pg_recevexlog will indeed fsync() 10 times per second. There's
nothing wrong with that.

In asynchronous mode, only fsync whenever switching to new segment.

>> Yeah. Or rather, add a new message type, to indicate the
>> synchronous/asynchronous status.
>
> What kind of 'message type' we have to add ?
>
> Do we need to separate 'w' into two types ? synchronous and asynchronous ?
>
> OR
>
> Add a new message type, kind of 'notify synchronous',
> and inform pg_receivexlog of synchronous status when it connect to the server.

Better to add a new "notify" message type. And pg_recevexlog should be
prepared to receive it at any time. The status might change on the fly,
if the server's configuration is reloaded.

- Heikki



--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
f***@pm.nttdata.co.jp
2014-10-09 09:42:51 UTC
Permalink
> >>> If we remove --fsync-interval, resoponse time to user will not be
> delay.
> >>> Although, fsync will be executed multiple times in a short period.
> >>> And there is no way to solve the problem without --fsync-interval,
> >>> what
> >> should we do about it?
> >>
> >> I'm sorry, I didn't understand that.
> >
> > Here is an example.
> > When WAL is sent at 100ms intervals, fsync() is executed 10 times per
> second.
> > If --fsync-interval is set by 1 second, we have to wait SQL
> responce(kind of making WAL record) for 1 second, though, fsync() won't
> be executed several times in 1 second.
> > I think --fsync-interval meets the demands of people who wants to
> restrain fsync() happens for several time in short period, but what do
> you think?
> > Is it ok to delete --fsync-interval ?
>
> I still don't see the problem.
>
> In synchronous mode, pg_receivexlog should have similar logic as
> walreceiver does. It should read as much WAL from the socket as it can
> without blocking, and fsync() and send reply after that. And also fsync
> whenever switching to new segment. If the master sends WAL every 100ms,
> then pg_recevexlog will indeed fsync() 10 times per second. There's
> nothing wrong with that.
>
> In asynchronous mode, only fsync whenever switching to new segment.
>
> >> Yeah. Or rather, add a new message type, to indicate the
> >> synchronous/asynchronous status.
> >
> > What kind of 'message type' we have to add ?
> >
> > Do we need to separate 'w' into two types ? synchronous and
> asynchronous ?
> >
> > OR
> >
> > Add a new message type, kind of 'notify synchronous', and inform
> > pg_receivexlog of synchronous status when it connect to the server.
>
> Better to add a new "notify" message type. And pg_recevexlog should be
> prepared to receive it at any time. The status might change on the fly,
> if the server's configuration is reloaded.

Thanks for the reply.

> In synchronous mode, pg_receivexlog should have similar logic as walreceiver does.

OK. I understand that removing --fsync-interval has no problem.

> Better to add a new "notify" message type. And pg_recevexlog should be prepared to receive it at any time. The status might change on the fly, if the server's configuration is reloaded.

OK. I'll consider it.

Regards,

--
Furuya Osamu

--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hac
Fujii Masao
2014-10-10 08:28:49 UTC
Permalink
On Thu, Oct 9, 2014 at 6:42 PM, <***@pm.nttdata.co.jp> wrote:
>> >>> If we remove --fsync-interval, resoponse time to user will not be
>> delay.
>> >>> Although, fsync will be executed multiple times in a short period.
>> >>> And there is no way to solve the problem without --fsync-interval,
>> >>> what
>> >> should we do about it?
>> >>
>> >> I'm sorry, I didn't understand that.
>> >
>> > Here is an example.
>> > When WAL is sent at 100ms intervals, fsync() is executed 10 times per
>> second.
>> > If --fsync-interval is set by 1 second, we have to wait SQL
>> responce(kind of making WAL record) for 1 second, though, fsync() won't
>> be executed several times in 1 second.
>> > I think --fsync-interval meets the demands of people who wants to
>> restrain fsync() happens for several time in short period, but what do
>> you think?
>> > Is it ok to delete --fsync-interval ?
>>
>> I still don't see the problem.
>>
>> In synchronous mode, pg_receivexlog should have similar logic as
>> walreceiver does. It should read as much WAL from the socket as it can
>> without blocking, and fsync() and send reply after that. And also fsync
>> whenever switching to new segment. If the master sends WAL every 100ms,
>> then pg_recevexlog will indeed fsync() 10 times per second. There's
>> nothing wrong with that.
>>
>> In asynchronous mode, only fsync whenever switching to new segment.
>>
>> >> Yeah. Or rather, add a new message type, to indicate the
>> >> synchronous/asynchronous status.
>> >
>> > What kind of 'message type' we have to add ?
>> >
>> > Do we need to separate 'w' into two types ? synchronous and
>> asynchronous ?
>> >
>> > OR
>> >
>> > Add a new message type, kind of 'notify synchronous', and inform
>> > pg_receivexlog of synchronous status when it connect to the server.
>>
>> Better to add a new "notify" message type. And pg_recevexlog should be
>> prepared to receive it at any time. The status might change on the fly,
>> if the server's configuration is reloaded.
>
> Thanks for the reply.
>
>> In synchronous mode, pg_receivexlog should have similar logic as walreceiver does.
>
> OK. I understand that removing --fsync-interval has no problem.

+1 for adding something like --synchronous option. To me,
it sounds walreceiver-compatible mode rather than synchronous.

>> Better to add a new "notify" message type. And pg_recevexlog should be prepared to receive it at any time. The status might change on the fly, if the server's configuration is reloaded.
>
> OK. I'll consider it.

I don't think that's good idea because it prevents us from using
pg_receivexlog as async walreceiver (i.e., received WAL data is
fsynced and feedback is sent back to the server soon,
but transaction commit in the server doesn't wait for the feedback).

Regards,

--
Fujii Masao


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
f***@pm.nttdata.co.jp
2014-10-14 09:44:57 UTC
Permalink
> >> >>> If we remove --fsync-interval, resoponse time to user will not
> be
> >> delay.
> >> >>> Although, fsync will be executed multiple times in a short period.
> >> >>> And there is no way to solve the problem without
> >> >>> --fsync-interval, what
> >> >> should we do about it?
> >> >>
> >> >> I'm sorry, I didn't understand that.
> >> >
> >> > Here is an example.
> >> > When WAL is sent at 100ms intervals, fsync() is executed 10 times
> >> > per
> >> second.
> >> > If --fsync-interval is set by 1 second, we have to wait SQL
> >> responce(kind of making WAL record) for 1 second, though, fsync()
> >> won't be executed several times in 1 second.
> >> > I think --fsync-interval meets the demands of people who wants to
> >> restrain fsync() happens for several time in short period, but what
> >> do you think?
> >> > Is it ok to delete --fsync-interval ?
> >>
> >> I still don't see the problem.
> >>
> >> In synchronous mode, pg_receivexlog should have similar logic as
> >> walreceiver does. It should read as much WAL from the socket as it
> >> can without blocking, and fsync() and send reply after that. And also
> >> fsync whenever switching to new segment. If the master sends WAL
> >> every 100ms, then pg_recevexlog will indeed fsync() 10 times per
> >> second. There's nothing wrong with that.
> >>
> >> In asynchronous mode, only fsync whenever switching to new segment.
> >>
> >> >> Yeah. Or rather, add a new message type, to indicate the
> >> >> synchronous/asynchronous status.
> >> >
> >> > What kind of 'message type' we have to add ?
> >> >
> >> > Do we need to separate 'w' into two types ? synchronous and
> >> asynchronous ?
> >> >
> >> > OR
> >> >
> >> > Add a new message type, kind of 'notify synchronous', and inform
> >> > pg_receivexlog of synchronous status when it connect to the server.
> >>
> >> Better to add a new "notify" message type. And pg_recevexlog should
> >> be prepared to receive it at any time. The status might change on the
> >> fly, if the server's configuration is reloaded.
> >
> > Thanks for the reply.
> >
> >> In synchronous mode, pg_receivexlog should have similar logic as
> walreceiver does.
> >
> > OK. I understand that removing --fsync-interval has no problem.
>
> +1 for adding something like --synchronous option. To me,
> it sounds walreceiver-compatible mode rather than synchronous.
>
> >> Better to add a new "notify" message type. And pg_recevexlog should
> be prepared to receive it at any time. The status might change on the
> fly, if the server's configuration is reloaded.
> >
> > OK. I'll consider it.
>
> I don't think that's good idea because it prevents us from using
> pg_receivexlog as async walreceiver (i.e., received WAL data is fsynced
> and feedback is sent back to the server soon, but transaction commit in
> the server doesn't wait for the feedback).

Thanks for the comment.

> it prevents us from using pg_receivexlog as async walreceiver.

Yes. If sync or async is switched by message,
in case pg_receivexlog is async, it won't work like a walreceiver.

User don't have to set options if synchronous status can be distinguished by message, though they cannot specify the action.

> adding something like --synchronous option
Integrate required options for sync like above, is more appropriate ?

Regards,

--
Furuya Osamu


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailp
Simon Riggs
2014-10-16 08:12:06 UTC
Permalink
On 10 October 2014 09:28, Fujii Masao <***@gmail.com> wrote:

>>> In synchronous mode, pg_receivexlog should have similar logic as walreceiver does.
>>
>> OK. I understand that removing --fsync-interval has no problem.
>
> +1 for adding something like --synchronous option. To me,
> it sounds walreceiver-compatible mode rather than synchronous.
>
>>> Better to add a new "notify" message type. And pg_recevexlog should be prepared to receive it at any time. The status might change on the fly, if the server's configuration is reloaded.
>>
>> OK. I'll consider it.
>
> I don't think that's good idea because it prevents us from using
> pg_receivexlog as async walreceiver (i.e., received WAL data is
> fsynced and feedback is sent back to the server soon,
> but transaction commit in the server doesn't wait for the feedback).

Sync rep works by setting parameters on the master. Standby servers
send replies by default, though you can turn replies off.

pg_receivexlog should work the same, but can't do this because it
doesn't know the fsync position unless it fsyncs.

So its not appropriate to have an option called "--synchronous" in the
same way that there is no parameter called "synchronous" on the
standby, for good reason.

A new parameter to send feedback should be called --feedback
A second parameter to decide whether to fsync should be called --fsync

if (feedback && fsync)
send fsynced LSN
else if (feedback)
send received LSN
; /* else send no feedback */

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
f***@pm.nttdata.co.jp
2014-10-17 08:55:42 UTC
Permalink
> >>> In synchronous mode, pg_receivexlog should have similar logic as
> walreceiver does.
> >>
> >> OK. I understand that removing --fsync-interval has no problem.
> >
> > +1 for adding something like --synchronous option. To me,
> > it sounds walreceiver-compatible mode rather than synchronous.
> >
> >>> Better to add a new "notify" message type. And pg_recevexlog should
> be prepared to receive it at any time. The status might change on the
> fly, if the server's configuration is reloaded.
> >>
> >> OK. I'll consider it.
> >
> > I don't think that's good idea because it prevents us from using
> > pg_receivexlog as async walreceiver (i.e., received WAL data is
> > fsynced and feedback is sent back to the server soon, but transaction
> > commit in the server doesn't wait for the feedback).
>
> Sync rep works by setting parameters on the master. Standby servers send
> replies by default, though you can turn replies off.
>
> pg_receivexlog should work the same, but can't do this because it doesn't
> know the fsync position unless it fsyncs.
>
> So its not appropriate to have an option called "--synchronous" in the
> same way that there is no parameter called "synchronous" on the standby,
> for good reason.
>
> A new parameter to send feedback should be called --feedback A second
> parameter to decide whether to fsync should be called --fsync
>
> if (feedback && fsync)
> send fsynced LSN
> else if (feedback)
> send received LSN
> ; /* else send no feedback */

Thanks for the comment.

The patch cannot be applied to HEAD cleanly so I updated.

>So its not appropriate to have an option called "--synchronous" in the same way that there is no >parameter called "synchronous" on the standby, for good reason.

In case of gathering options to one option,
change the name "--synchronous" to other name solves the problem ?

>A new parameter to send feedback should be called --feedback
>A second parameter to decide whether to fsync should be called --fsync

I think keep using "--reply-fsync" and "--fsync-interval" is better than make new options.
Thought?

Regards,

--
Furuya Osamu
Simon Riggs
2014-10-17 10:59:33 UTC
Permalink
On 17 October 2014 09:55, <***@pm.nttdata.co.jp> wrote:

>>A new parameter to send feedback should be called --feedback

>>A second parameter to decide whether to fsync should be called --fsync
>
> I think keep using "--reply-fsync" and "--fsync-interval" is better than make new options.
> Thought?

We already have hot_standby_feedback, so using the name feedback is best idea.

I am suggesting that we send feedback even if we do not fsync, to
allow the master to track our progress. Hence the name of the second
parameter was just fsync.

So both names were suggested because of links to those terms already
being used for similar reasons elsewhere in Postgres.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Heikki Linnakangas
2014-10-22 13:26:00 UTC
Permalink
On 10/17/2014 01:59 PM, Simon Riggs wrote:
> On 17 October 2014 09:55, <***@pm.nttdata.co.jp> wrote:
>
>>> A new parameter to send feedback should be called --feedback
>
>>> A second parameter to decide whether to fsync should be called --fsync
>>
>> I think keep using "--reply-fsync" and "--fsync-interval" is better than make new options.
>> Thought?
>
> We already have hot_standby_feedback, so using the name feedback is best idea.
>
> I am suggesting that we send feedback even if we do not fsync, to
> allow the master to track our progress. Hence the name of the second
> parameter was just fsync.
>
> So both names were suggested because of links to those terms already
> being used for similar reasons elsewhere in Postgres.

We seem to be going in circles. You suggested having two options,
--feedback, and --fsync, which is almost exactly what Furuya posted
originally. I objected to that, because I think that user interface is
too complicated. Instead, I suggested having just a single option called
--synchronous, or even better, have no option at all and have the server
tell the client if it's participating in synchronous replication, and
have pg_receivexlog automatically fsync when it is, and not otherwise
[1]. That way you don't need to expose any new options to the user. What
did you think of that idea?

[1] http://www.postgresql.org/message-id/***@vmware.com

- Heikki



--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Simon Riggs
2014-10-22 13:47:16 UTC
Permalink
On 22 October 2014 14:26, Heikki Linnakangas <***@vmware.com> wrote:

> We seem to be going in circles. You suggested having two options,
> --feedback, and --fsync, which is almost exactly what Furuya posted
> originally. I objected to that, because I think that user interface is too
> complicated. Instead, I suggested having just a single option called
> --synchronous, or even better, have no option at all and have the server
> tell the client if it's participating in synchronous replication, and have
> pg_receivexlog automatically fsync when it is, and not otherwise [1]. That
> way you don't need to expose any new options to the user. What did you think
> of that idea?

Sorry, if we're going in circles.

Yes, I like the idea.

The master setting of synchronous_standby_names defines which
standbys/rep clients will have their feedback used to release waiting
COMMITs. That uses application_name, which is set at the replication
client connection. (That has a default value, but lets assume the user
sets this). So when a replication client connects, the WALSender knows
its priority, as defined in sync_standby_names.

I guess we can have WALSender send a protocol message to the client
every time the sync priority changes (as a result of a changed value
of sync_standby_names). Which is the function SyncRepInitConfig()

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Fujii Masao
2014-10-23 14:39:58 UTC
Permalink
On Wed, Oct 22, 2014 at 10:47 PM, Simon Riggs <***@2ndquadrant.com> wrote:
> On 22 October 2014 14:26, Heikki Linnakangas <***@vmware.com> wrote:
>
>> We seem to be going in circles. You suggested having two options,
>> --feedback, and --fsync, which is almost exactly what Furuya posted
>> originally. I objected to that, because I think that user interface is too
>> complicated. Instead, I suggested having just a single option called
>> --synchronous

I'm OK with this. But the option name "synchronous" seems confusing
because pg_receivexlog with sync option can work as async standby.
So the better name is needed.

> or even better, have no option at all and have the server
>> tell the client if it's participating in synchronous replication, and have
>> pg_receivexlog automatically fsync when it is, and not otherwise [1]. That
>> way you don't need to expose any new options to the user. What did you think
>> of that idea?
>
> Sorry, if we're going in circles.
>
> Yes, I like the idea.
>
> The master setting of synchronous_standby_names defines which
> standbys/rep clients will have their feedback used to release waiting
> COMMITs. That uses application_name, which is set at the replication
> client connection. (That has a default value, but lets assume the user
> sets this). So when a replication client connects, the WALSender knows
> its priority, as defined in sync_standby_names.
>
> I guess we can have WALSender send a protocol message to the client
> every time the sync priority changes (as a result of a changed value
> of sync_standby_names). Which is the function SyncRepInitConfig()

Sorry, I'm going around in the circle. But I'd like to say again, I don't think
this is good idea. It prevents asynchronous pg_receivexlog from fsyncing
WAL data and sending feedbacks more frequently at all. They are useful,
for example, when we want to monitor the write location of asynchronous
pg_receivexlog in almost realtime. But if we adopt the idea, since feedback
cannot be sent soon in async mode, pg_stat_replication always returns
the not-up-to-date location.

Regards,

--
Fujii Masao


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Simon Riggs
2014-10-23 15:01:47 UTC
Permalink
On 23 October 2014 15:39, Fujii Masao <***@gmail.com> wrote:

> Sorry, I'm going around in the circle. But I'd like to say again, I don't think
> this is good idea. It prevents asynchronous pg_receivexlog from fsyncing
> WAL data and sending feedbacks more frequently at all. They are useful,
> for example, when we want to monitor the write location of asynchronous
> pg_receivexlog in almost realtime. But if we adopt the idea, since feedback
> cannot be sent soon in async mode, pg_stat_replication always returns
> the not-up-to-date location.

Why not send a message every 10 seconds when its not sync rep?

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Heikki Linnakangas
2014-10-23 15:11:58 UTC
Permalink
On 10/23/2014 06:01 PM, Simon Riggs wrote:
> On 23 October 2014 15:39, Fujii Masao <***@gmail.com> wrote:
>
>> Sorry, I'm going around in the circle. But I'd like to say again, I don't think
>> this is good idea. It prevents asynchronous pg_receivexlog from fsyncing
>> WAL data and sending feedbacks more frequently at all. They are useful,
>> for example, when we want to monitor the write location of asynchronous
>> pg_receivexlog in almost realtime. But if we adopt the idea, since feedback
>> cannot be sent soon in async mode, pg_stat_replication always returns
>> the not-up-to-date location.
>
> Why not send a message every 10 seconds when its not sync rep?

Or even after every write(). It's a tiny amount of network traffic anyway.

- Heikki



--
Sent via pgsql-hackers mailing list (pgsql-***@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Loading...