ntfs-3g data deduplication support in guestmount for creating file based back-ups on visualization platform
10:06 a.m.
Hello everybody,
I am using libguestfs-tools version 1.28.1 in a bunch of bash scripts to
make back-ups of the files on a visualisation platform.
I been trying to get read access to ntfs volumes with data deduplication
working.
The ntfs-3g guys developed some support and I have this working when
using the commands our back-up servers.
However when using the guestmount tool the read suport foor dedub is gone.
See this email for all the commands I use (working mount -t ntfs and
non-working guestmount commands).
https://sourceforge.net/p/ntfs-3g/mailman/message/35595392/
How can I see what version of ntfs-3g is used in guestmount and if it is
using the /usr/lib/x86_64-linux-gnu/ntfs-3g/ntfs-plugin-80000013.so
plugin that is on my back-up server?
Thank you in advance!
Kind regards,
Jelle de Jong
8:29 a.m.
On Tue, Jan 10, 2017 at 05:06:13PM +0100, Jelle de Jong wrote:
Hello everybody,
I am using libguestfs-tools version 1.28.1 in a bunch of bash
scripts to make back-ups of the files on a visualisation platform.
I been trying to get read access to ntfs volumes with data
deduplication working.
Do you have more information about data deduplication? It's the first
I've heard of it for NTFS (or indeed any ordinary Unix filesystem).
In fact I thought it was a feature of the block layer rather than the
filesystem. But anyway ...
The ntfs-3g guys developed some support and I have this working when
using the commands our back-up servers.
However when using the guestmount tool the read suport foor dedub is gone.
This is maybe not especially surprising. A lot of layers have to work
correctly in order for a feature to be passed through guestmount.
You've got (at least) kernel+FUSE on the host side (for guestmount),
the libguestfs API, the appliance kernel, and ntfs-3g in the
appliance. Getting discard support working involved a considerable
effort.
See this email for all the commands I use (working mount -t ntfs and
non-working guestmount commands).
https://sourceforge.net/p/ntfs-3g/mailman/message/35595392/
How can I see what version of ntfs-3g is used in guestmount and if
it is using the
/usr/lib/x86_64-linux-gnu/ntfs-3g/ntfs-plugin-80000013.so plugin
that is on my back-up server?
Probably helpful to refer to the diagram here:
http://libguestfs.org/guestfs-internals.1.html
ntfs-3g in your case is running in the "appliance" box in that
diagram.
The plugin file *should* be copied into the appliance, assuming that
it's part of the Debian ntfs-3g package and the Debian ntfs-3g package
isn't doing anything weird.
You can easily check using virt-rescue:
$ virt-rescue --ro -a windows.img
<rescue> ls -l /usr/lib/x86_64-linux-gnu/ntfs-3g/
<rescue> mount /dev/sda1 /sysroot
(and then use whatever ntfs commands are needed to check that the
plugin is being used)
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW
9:20 a.m.
Thank you Richard!
I will do some more work and testing on this tomorrow, and report back
with some more information you asked.
In the meanwhile, could you help me with information how I can upgrade
the ntfs-3g version (same as host) in the "appliance" and add the
plug-in library?
On 16/01/17 15:29, Richard W.M. Jones wrote:
The plugin file *should* be copied into the appliance, assuming that
it's part of the Debian ntfs-3g package and the Debian ntfs-3g package
isn't doing anything weird.
The plug-in is added manually and is not part of the ntfs-3g (yet), it
is manually downloaded and copied.
Thank you in advance!
Kind regards,
Jelle de Jong
9:43 a.m.
On Mon, Jan 16, 2017 at 04:20:14PM +0100, Jelle de Jong wrote:
Thank you Richard!
I will do some more work and testing on this tomorrow, and report
back with some more information you asked.
In the meanwhile, could you help me with information how I can
upgrade the ntfs-3g version (same as host) in the "appliance" and
add the plug-in library?
[...]
The plug-in is added manually and is not part of the ntfs-3g (yet),
it is manually downloaded and copied.
You can most likely drop a file into /usr/lib/guestfs/supermin.d/
which contains a single line, being the full path of the plugin
file that you want to add.
Actually you can use wildcards there too, so this should work:
echo '/usr/lib/x86_64-linux-gnu/ntfs-3g/ntfs-plugin-*.so' >
/usr/lib/guestfs/supermin.d/zz-ntfs-plugins
After that, do:
rm -rf /var/tmp/.guestfs-*
and rerun your libguestfs program (guestmount or whatever).
See "SUPERMIN APPLIANCES" section in the supermin(1) man page for
further information.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top
3:05 p.m.
Hi Richard,
Is there a way I can access the /var/log/syslog of an running guestmount
instance to get the grep ntfs-3g /var/log/syslog information?
I managed to get the plug-in to load in my questmount instances by using:
echo '/usr/lib/x86_64-linux-gnu/ntfs-3g/ntfs-plugin-*.so' >
/usr/lib/guestfs/supermin.d/zz-ntfs-plugins
rm -rf /var/tmp/.guestfs-*
Kind regards,
Jelle de Jong
On 16/01/17 16:43, Richard W.M. Jones wrote:
On Mon, Jan 16, 2017 at 04:20:14PM +0100, Jelle de Jong wrote:
> Thank you Richard!
>
> I will do some more work and testing on this tomorrow, and report
> back with some more information you asked.
>
> In the meanwhile, could you help me with information how I can
> upgrade the ntfs-3g version (same as host) in the "appliance" and
> add the plug-in library?
[...]
> The plug-in is added manually and is not part of the ntfs-3g (yet),
> it is manually downloaded and copied.
You can most likely drop a file into /usr/lib/guestfs/supermin.d/
which contains a single line, being the full path of the plugin
file that you want to add.
Actually you can use wildcards there too, so this should work:
echo '/usr/lib/x86_64-linux-gnu/ntfs-3g/ntfs-plugin-*.so' >
/usr/lib/guestfs/supermin.d/zz-ntfs-plugins
After that, do:
rm -rf /var/tmp/.guestfs-*
and rerun your libguestfs program (guestmount or whatever).
See "SUPERMIN APPLIANCES" section in the supermin(1) man page for
further information.
Rich.
3:18 p.m.
On Thu, Feb 09, 2017 at 10:05:36PM +0100, Jelle de Jong wrote:
Hi Richard,
Is there a way I can access the /var/log/syslog of an running
guestmount instance to get the grep ntfs-3g /var/log/syslog
information?
Not of a running guestmount, no, however if you're using the
API then you can use the debug "back door" to read dmesg and
log files. There is no documentation for it, but you can find
out what is available by browsing the sources:
https://github.com/libguestfs/libguestfs/blob/master/daemon/debug.c#L83
Something like:
const char *cmd[] = { "cat", "/var/log/syslog", NULL };
guestfs_debug (g, "sh", (char**)cmd);
or the equivalent in other language bindings.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine. Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
2842
days inactive
2872
days old
5 comments
2 participants
participants (2)
-
Jelle de Jong -
Richard W.M. Jones