Re: [Libguestfs] libnbd flamegraph Odd benchmark results
On Sun, May 30, 2021 at 09:27:18PM +0530, Abhay Raj Singh wrote:
The question (for you!) is how we can choose better settings, and
whether we should choose
In my opinion configuration resolution will be best if resolved in the
following order of increasing priority/weightage.
1. Auto-detect depending on the system parameters.
Auto-detection of best params is a must IMO as there's
enough diversity in server hardware that one size won't fit
all can be implemented without much effort(excluding benchmarking).
What are the inputs to this formula? I think the only ones we can
portably get are: number of cores, total memory. Free memory
possibly, although it's a slippery concept (is that "free" before or
after the page cache is freed?) Other inputs that could make a
difference, such as number of NUMA nodes, size of L2/L3 cache, core
clock rate, ... are difficult to get reliably and portably.
Still, if the formula is clear and understandable to users it could be
fine.
2. Configuration file: A yaml, toml, or any other simple format
- Load a different config file with *nbdcopy --file=custom-config.yaml*
- Maybe generate the default config file having values from the auto-detect
phase.
nbdcopy --generate-config
~/.config/nbdcopy/config.yaml (used to get values)
~/.config/nbdcopy/example.yaml (generated for reference from auto)
I think this part is a horrible idea for several reasons:
- Adds hidden state: As an end user you need to know the configuration
file exists. And if end users have problems we need them to send us
the configuration file, and my experience with virt-v2v tells me
that follow-up questions about bugs are rarely answered.
- Scripts will behave differently depending on where they are running.
- Any script or program consuming nbdcopy must disable the config file
processing. It will be completely unwanted for them to depend on
the random contents of a config file that could greatly affect (or
even break) the program.
- Adds much complexity to the implementation of the tool.
- Yaml etc are awful formats for configuration files in general.
Rich.
3. CLI params
A quick way to override configuration file params
ex: functionally same
`nbdcopy --requests=4 --config=custom-config.yaml`
`nbdcopy --config=custom-config.yaml --requests=4`
The flow of initialization:
Read config file ----override ---> CLI params --- missing params -->
auto-detect fill
or
easier implementation:
auto-detect fill --- override ---> config file --- override ---> CLI params
The rationale for choosing configuration file method:
- Easier to use avoiding long commands (less typing, scripts easier to read)
- Easy switch between multiple configurations and override them.
- Documentation and configuration in the same place
So easier for someone else to understand why certain parameters were chosen.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org