11:39 a.m.
Here is an example of a failure with the previous dependency
calculation:
$ make -j5
make: Entering directory '/home/rjones/d/libguestfs/common/mlstdutils'
OCAMLOPT guestfs_config.cmx
CC libmlstdutils_a-dummy.o
OCAMLOPT libdir.cmx
OCAMLCMI stringMap.cmi
OCAMLCMI stringSet.cmi
OCAMLCMI guestfs_config.cmi
OCAMLCMI std_utils.cmi
OCAMLC guestfs_config.cmo
OCAMLC libdir.cmo
OCAMLC stringMap.cmo
OCAMLC stringSet.cmo
OCAMLOPT stringMap.cmx
OCAMLOPT stringSet.cmx
OCAMLOPT std_utils.cmx
OCAMLC std_utils.cmo
ocamlfind ocamlc -package str,unix -I . -a guestfs_config.cmo libdir.cmo stringMap.cmo
stringSet.cmo std_utils.cmo -o mlstdutils.cma
ocamlfind ocamlopt -package str,unix -I . -a guestfs_config.cmx libdir.cmx stringMap.cmx
stringSet.cmx std_utils.cmx -o mlstdutils.cmxa
AR libmlstdutils.a
File "_none_", line 1:
Error: Files std_utils.cmx and guestfs_config.cmx
make inconsistent assumptions over interface Guestfs_config
make: *** [Makefile:2523: mlstdutils.cmxa] Error 2
make: Leaving directory '/home/rjones/d/libguestfs/common/mlstdutils'
What seems to be happening is that there is a rule:
std_utils.cmx : guestfs_config.cmi guestfs_config.cmx [...]
In this case, make chose to build guestfs_config.cmx and
guestfs_config.cmi in parallel (see the first 5 rules above). However
building guestfs_config.cmx also creates guestfs_config.cmi
(implicitly - this is not expressed in make dependencies, and make
doesn't "know" that guestfs_config.cmi has already been created).
Unfortunately the OCaml compiler doesn't create output files
atomically. Worse than that, it creates an intermediate version of
the output file, reads it back and then creates the final version. It
seems if the build of std_utils.cmi reads this intermediate version.
In any case, Std_utils sees the wrong hash of the Guestfs_config
module.
The above only happens where we have a *.ml file without a
corresponding *.mli file. That is because if there is a *.mli file,
ocamldep generates slightly different dependencies:
guestfs_config.cmx [...] : guestfs_config.cmi guestfs_config.ml
std_utils.cmx : guestfs_config.cmi guestfs_config.cmx [...]
std_utils.cmx still depends on both files, but there is an extra rule
which ensures that guestfs_config.cmx isn't built in parallel with
guestfs_config.cmi.
I tested this change by running this command:
$ while ( rm common/mlstdutils/.depend; make -C common/mlstdutils/ clean && make
-C common/mlstdutils/ ) >& /tmp/log; do echo -n .; done
Before the change it would fail after about 100 iterations. After the
change it ran for 10000s iterations and did not fail ever.
Updates commit 6d0ad49d5e3415de45a6ccdf1ec6de55e1e8384f.
---
subdir-rules.mk | 7 -------
1 file changed, 7 deletions(-)
diff --git a/subdir-rules.mk b/subdir-rules.mk
index 857afa0ab..9feb115b2 100644
--- a/subdir-rules.mk
+++ b/subdir-rules.mk
@@ -79,15 +79,8 @@ guestfs_am_v_jar = $(guestfs_am_v_jar_@AM_V@)
guestfs_am_v_jar_ = $(guestfs_am_v_jar_@AM_DEFAULT_V@)
guestfs_am_v_jar_0 = @echo " JAR " $@;
-# We must always choose the .mli.cmi rule over the .ml.cmi rule if the
-# .mli file exists. The .mli.cmi rule is listed first because:
-# "If more than one pattern rule has the shortest stem, make will
-# choose the first one found in the makefile."
-# (https://www.gnu.org/software/make/manual/make.html#Pattern-Match)
%.cmi: %.mli
$(guestfs_am_v_ocamlcmi)$(OCAMLFIND) ocamlc $(OCAMLFLAGS) $(OCAMLPACKAGES) -c $< -o
$@
-%.cmi: %.ml
- $(guestfs_am_v_ocamlcmi)$(OCAMLFIND) ocamlc $(OCAMLFLAGS) $(OCAMLPACKAGES) -c $< -o
$@
%.cmo: %.ml
$(guestfs_am_v_ocamlc)$(OCAMLFIND) ocamlc $(OCAMLFLAGS) $(OCAMLPACKAGES) -c $< -o $@
if HAVE_OCAMLOPT
--
2.13.2