Christopher B. Browne's Home Page
cbbrowne@acm.org

4.4. Log Shipping - Slony-I with Files

Slony-I has the ability to serialize the updates to go out into log files that can be kept in a spool directory.

The spool files could then be transferred via whatever means was desired to a "slave system," whether that be via FTP, rsync, or perhaps even by pushing them onto a 1GB "USB key" to be sent to the destination by clipping it to the ankle of some sort of "avian transport" system.

There are plenty of neat things you can do with a data stream in this form, including:

Q: Where are the "spool files" for a subscription set generated?
Q: What takes place when a SLONIK FAILOVER(7)/ SLONIK MOVE SET(7) takes place?
Q: What if we run out of "spool space"?
Q: How do we set up a subscription?
Q: What are the limitations of log shipping?

Q: Where are the "spool files" for a subscription set generated?

A: Any slon subscriber node can generate them by adding the -a option.

Note

Notice that this implies that in order to use log shipping, you must have at least one subscriber node.

Q: What takes place when a SLONIK FAILOVER(7)/ SLONIK MOVE SET(7) takes place?

A: Nothing special. So long as the archiving node remains a subscriber, it will continue to generate logs.

A:

If the archiving node becomes the origin, on the other hand, it will continue to generate logs.

Q: What if we run out of "spool space"?

A: The node will stop accepting SYNCs until this problem is alleviated. The database being subscribed to will also fall behind.

Q: How do we set up a subscription?

A: The script in tools called slony1_dump.sh is a shell script that dumps the "present" state of the subscriber node.

You need to start the slon for the subscriber node with logging turned on. At any point after that, you can run slony1_dump.sh, which will pull the state of that subscriber as of some SYNC event. Once the dump completes, all the SYNC logs generated from the time that dump started may be added to the dump in order to get a "log shipping subscriber."

Q: What are the limitations of log shipping?

A: In the initial release, there are rather a lot of limitations. As releases progress, hopefully some of these limitations may be alleviated/eliminated.

A: The log shipping functionality amounts to "sniffing" the data applied at a particular subscriber node. As a result, you must have at least one "regular" node; you cannot have a cluster that consists solely of an origin and a set of "log shipping nodes.".

A: The "log shipping node" tracks the entirety of the traffic going to a subscriber. You cannot separate things out if there are multiple replication sets.

A: The "log shipping node" presently only fully tracks SYNC events. This should be sufficient to cope with some changes in cluster configuration, but not others.

A number of event types are handled in such a way that log shipping copes with them:

  • SYNC events are, of course, handled.

  • DDL_SCRIPT is handled.

  • UNSUBSCRIBE_SET

    This event, much like SUBSCRIBE_SET is not handled by the log shipping code. But its effect is, namely that SYNC events on the subscriber node will no longer contain updates to the set.

    Similarly, SET_DROP_TABLE, SET_DROP_SEQUENCE, SET_MOVE_TABLE, SET_MOVE_SEQUENCE, DROP_SET, MERGE_SET, SUBSCRIBE_SET will be handled "apropriately".

  • The various events involved in node configuration are irrelevant to log shipping: STORE_NODE, ENABLE_NODE, DROP_NODE, STORE_PATH, DROP_PATH, STORE_LISTEN, DROP_LISTEN

  • Events involved in describing how particular sets are to be initially configured are similarly irrelevant: STORE_SET, SET_ADD_TABLE, SET_ADD_SEQUENCE, STORE_TRIGGER, DROP_TRIGGER,

A: It would be nice to be able to turn a "log shipped" node into a fully communicating Slony-I node that you could failover to. This would be quite useful if you were trying to construct a cluster of (say) 6 nodes; you could start by creating one subscriber, and then use log shipping to populate the other 4 in parallel.

This usage is not supported, but presumably one could take an application outage and promote the log-shipping node to a normal slony node with the OMIT COPY option of SUBSCRIBE SET.

4.4.1. Usage Hints

Note

Here are some more-or-less disorganized notes about how you might want to use log shipping...

4.4.2. find-triggers-to-deactivate.sh

It was once pointed out ( Bugzilla bug #19) that the dump of a schema may include triggers and rules that you may not wish to have running on the log shipped node.

The tool tools/find-triggers-to-deactivate.sh was created to assist with this task. It may be run against the node that is to be used as a schema source, and it will list the rules and triggers present on that node that may, in turn need to be deactivated.

It includes logtrigger and denyaccess triggers which will may be left out of the extracted schema, but it is still worth the Gentle Administrator verifying that such triggers are kept out of the log shipped replica.

4.4.3. slony_logshipper Tool

As of version 1.2.12, Slony-I has a tool designed to help apply logs, called slony_logshipper. It is run with three sorts of parameters:

Google

If this was useful, let others know by an Affero rating

Contact me at cbbrowne@acm.org