Q: Why should lvmetad cache foreign VGs?
A: It's the most useful behavior in the "steady state".

How to arrive at that conclusion.
Four code configurations to consider, each in two different circumstances.

configurations:

1. lvm not using lvmetad
2. lvm using lvmetad and lvmlockd
3. lvm using lvmetad, and lvmetad does not cache foreign VGs
   (Not currently implemented.)
4. lvm using lvmetad, and lvmetad caches foreign VGs

circumstances:

A. steady state: PVs are not added or removed to/from foreign VGs
B. transient state: PVs are added or removed to/from foreign VGs

combinations:

1.A. A PV is correctly shown in the foreign VG.
1.B. A PV is correctly shown in the foreign VG.

The most accurate representation, at the cost of always scanning disks.


2.A. A PV is correctly shown in the foreign VG.
2.B. A PV is correctly shown in the foreign VG.

The most accurate representation, at the cost of using lvmlockd.


3.A. A PV in a foreign VG is shown as unused.
3.B. A PV in a foreign VG is shown as unused.

If lvmetad ignores foreign VGs and does not cache them, the PVs in the
foreign VGs appear to be unused.  This largely defeats the purpose of
system_id, which is meant to treat VGs/PVs as foreign instead of free
(albeit imperfectly, see below.)


4.A. A PV is correctly shown in the foreign VG.
4.B. A PV is not correctly shown in the foreign VG.

This avoids the cost of always scanning disks, and avoids the cost of
using lvmlockd.  The steady state 4.A. is an improvement over the steady
state 3.A.  When the steady state is the common case, this is a big
advantage.  When the steady state is *not* the common case, the foreign VG
concept is not as useful (if shared devices are this dynamic, lvmlockd
should be considered.)

The limitations related to the transient state 4.B. are explained in
lvmsystemid(7), along with how to handle it.  The specific inaccuracies
possible in 4.B. are:

. PV is shown as belonging to a foreign VG, but is actually unused.
. PV is shown as unused, but actually belongs to a foreign VG.

To resolve the inaccuracies in the transient state (4.B.), and return the
system to an accurate steady state (4.A.), the disks need to be scanned,
which updates lvmetad.  The scanning/updating is a manual step, i.e.
running 'pvscan --cache', which by definition scans disks and updates
lvmetad.

--

The --foreign command line option for report/display commands
(vgs/lvs/pvs/vgdisplay/lvdisplay/pvdisplay) is not directly related to
whether or not lvmetad caches foreign VGs.

By default, foreign VGs are silently ignored and not printed by these
commands.  However, when the --foreign option is used, these commands do
produce output about foreign VGs.

(When --foreign is not used, and the command specifically requests a
foreign VG by name, an error is produced about not accessing foreign VGs,
and the foreign VG is not displayed.)

The decision to report/display foreign VGs or not is independent of
whether lvmetad is caching those VGs.  When lvmetad is caching the foreign
VG, a report/display command run with --foreign will scan disks to read
the foreign VG and give the most up to date version of it (the copy of the
foreign VG in lvmetad may be out of date due to changes to the VG by the
foreign host.)