Hello all,

we've run into the same problem of the broken connection between master and slave not being detected by the slave. We are using some older version of GTM so my question is if there is something new regarding this issue in newer versions? For exmaple optional keep alive support or other solution?

Jan B.

Dne čtvrtek 13. října 2016 v 1:25:24 UTC+2 uživatel attila....@gmail.com napsal:
> Bhaskar,
>
>
> I believe I have a fairly good handle on why the replication stalls.
> If you live "in the cloud" then you may have very (actually close to zero) influence on the network gear and its operation between your sites. So I learned that if reboot our network equipment or the service provider decides to rearrange the network then we end up in a situation where the source realizes that the connection is broken and tries to reconnect. Unfortunately the receiver does not and keeps the connection open - with netstat one may just see it. Now since the receiver thinks it has a valid connection it refuses the connection attempt from the source. Nice deadlock. It can stay that way for a considerable time - even days. We are in the process to upgrade to GTM version 6 and I hope this behavior is fixed.
>
> Well, if not then I can at least monitor it with the command you provided.. Thank you for that. I was checking the documentation if something similar exists for the receiver side but no luck.
>
> I was having a second thought about Infohub and I wonder where the Infohub database should reside ? The documentations says: " ...one InfoHub can monitor multiple data sources, and a single data source can be monitored by multiple InfoHubs. "
> But it is not clear for me neither from the text nor from the diagram if the datasource and the Infohub DB should reside in the same machine.
>
>
> Attila
> On Monday, October 10, 2016 at 4:14:45 PM UTC+2, K.S. Bhaskar wrote:
> > On Friday, October 7, 2016 at 2:58:19 PM UTC-4, Attila Csikai wrote:
> > > On Friday, October 7, 2016 at 4:04:42 PM UTC+2, K.S. Bhaskar wrote:
> > > > On Thursday, October 6, 2016 at 7:49:47 PM UTC-4, Attila Csikai wrote:
> > > > > Hi,
> > > > >
> > > > >
> > > > >
> > > > > I am working on monitoring GTM replication and I have two questions:
> > > > >
> > > > > 1. I wonder how one can detect a situation when the replication (data flow) stops but both the source and the receiver server processes are out there.
> > > > > mupip replicate -source/receiver -checkhealth shows no problem and also
> > > > > mupip replicate -source/receiver -showbacklog looks OK.
> > > > > (Well, after a -short- while the source side starts to accumulate backlog, but that is all.)
> > > > >
> > > > > The only place to indicate an error is the source server log containing hard and soft connection attempts.
> > > > >
> > > > > Is the log the only place to detect disconnect ?
> > > > >
> > > > > 2. My monitoring agent is a C program and I have identified the following options to get information about replication status (short of mining the log):
> > > > > a, running "mupip replicate -source/receiver ... " by opening a pipe -popen- and grabbing the output of the command,
> > > > > b, %PEEKBYNAME() may provide some information.
> > > > >
> > > > > Unfortunately option "a" is somewhat resource intensive carrying a significant overhead. Option "b" is only available in version 6.3 and I either spin up a MUMPS process every time to read the information (significant overhead) or keep one alive to periodically emit the required info -then I need to have IPC and manage this process as well.
> > > > >
> > > > > The best would be to invoke mupip functionality as a shared library function but it is not possible as far as I know.
> > > > >
> > > > > Are there other possibilities?
> > > > >
> > > > > Thank you,
> > > > > Attila
> > > >
> > > > Attila --
> > > >
> > > > It is not possible to call into mupip as a shared library. However, you don't have to spin up a mumps process every time - you can call a MUMPS routine from your C program (look at Chapter 11 - Integrating External Routines - of the Programmers Guide; it even has a downloadable working example of calling M code from C code).
> > > >
> > > > Did you consider using InfoHub for your monitoring?
> > > >
> > > > Regards
> > > > -- Bhaskar
> > >
> > >
> > > Bashkar,
> > >
> > >
> > > Thank you for bringing up the CI interface. Actually I know it and used it (previously) but obviously overlooked this possibility.
> > >
> > > I had superficial knowledge about Infohub but now took a more serious look.
> > > However as I only have to do data gathering for an existing monitoring tool I doubt there is room for Infohub as a whole.
> > > Nevertheless the IHRRPCmdXXXXXXXXX.m routine is very interesting.
> > >
> > > But even in that routine I do not seem to find the answer to my first question about how to detect a broken replication dataflow. I wonder if you can give a hint.
> > >
> > >
> > > Thank you,
> > > Attila
> >
> > Attila --
> >
> > You seem to have a situation where a Source Server and a Receiver Server have a connection (or had a connection), which is not a normal situation. You will need to diagnose what went wrong, and for that you will need to look at the logs.
> >
> > For monitoring, perhaps you can try a command like: mupip replic -source -jnlpool -show | & grep "Processing State"
> >
> > An output like: SRC # 0 : Processing State WAITING_FOR_CONNECTION
> > indicates the source server has not yet connected with a receiver.
> >
> > Once connected, you can expect to see: SRC # 0 : Processing State SENDING_JNLRECS
> >
> > Below is the list of possible values.
> >
> > "DUMMY_STATE",
> > "START",
> > "WAITING_FOR_CONNECTION",
> > "WAITING_FOR_RESTART",
> > "SEARCHING_FOR_RESTART",
> > "SENDING_JNLRECS",
> > "WAITING_FOR_XON",
> > "CHANGING_MODE"
> >
> > Regards
> > -- Bhaskar