4 a.m.
I did one with ntbackup and restore a few months ago and ran into problems
with the virtual cpu not matching the virtual bios. That one was in a
libvirt environment and that VM will retire soon. But with the current
project, these VMs will be around a while.
Sent from my Droid Charge on Verizon 4GLTE
Matthew Booth <mbooth(a)redhat.com> wrote:
On 12/02/2011 03:12 AM, Greg Scott wrote:
Hey Matt -
With a new release of virt-v2v a while away, what if we were to try a
P2V by hand? I'm thinking of maybe provisioning a Windows RHEV VM,
doing an ntbackup of the source physical machine and doing a bare metal
restore on the VM. I've done it that way before with mixed results.
We have that hack I put together to load the viostor block driver stuff
on the source machine so the target VM will boot. Are there any HAL
kinds of things or other hardware/firmware issues we need to also be
thinking about?
Windows mostly fixes itself. If it boots and you don't see any missing
drivers in Device Manager, you're probably good to go.
Matt
--
Matthew Booth, RHCA, RHCSS
Red Hat Engineering, Virtualisation Team
GPG ID: D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490
3:18 p.m.
New subject: P2Vs seem to require a very robust Ethernet
Our track record for P2Vs isn't very good so far; we've had one run to
completion in about 8 or 9 tries so far. We've tried source systems
with different versions of Windows, all but one failed. The one success
was a test Windows XP machine with one hard drive.
We boot a Windows system from the CD, start up a P2V, it runs for an
apparently random length of time, then the P2V percentage just stops
somewhere. The other day was 13 percent. Today it's 8 percent.
Eventually, virt-p2v-server on the conversion server also stops and I
see a message in the log file that says, " virt-v2v: Error receiving
data:".
Is there anything we can look at from an alternate console on the source
server booted from that CD?
Thanks
- Greg
5:57 p.m.
New subject: P2Vs seem to require a very robust Ethernet
After 5 or 6 hours, we aborted today's attempt, disconnected the 2nd HDD
(sdb) from the original host, and started up a new one.
Since all we have to go on are guesses - there was a bug with the
Windows virtio network driver where the driver would go offline during
bulk copies. I bring this up because our P2V conversion server is a
Fedora VM, probably using the Linux virtio driver. Was there a similar
bug with the Linux VirtIO driver?
One possible sequence of events:
- Data flies fast and furious in and out of the conversion server VM to
the NFS server.
- The conversion server VM NIC goes offline.
- The source host eventually hangs or dies.
- virt-p2v-server eventually gives up and writes a cryptic log message.
Another possible sequence of events:
- The source server starts copying.
- The source server runs into a problem and either hangs or dies.
- Virt-p2v-server on the conversion server eventually times out waiting
for data.
What do you think about trying a physical conversion server, possibly
using itself as the NFS Server and RHEV Export domain?
- Greg
-----Original Message-----
From: libguestfs-bounces(a)redhat.com
[mailto:libguestfs-bounces@redhat.com] On Behalf Of Greg Scott
Sent: Friday, December 02, 2011 3:18 PM
To: Matthew Booth; Richard W.M. Jones; libguestfs(a)redhat.com
Cc: Fredy Hernandez; Joey Bertalan
Subject: Re: [Libguestfs] P2Vs seem to require a very robust Ethernet
Our track record for P2Vs isn't very good so far; we've had one run to
completion in about 8 or 9 tries so far. We've tried source systems
with different versions of Windows, all but one failed. The one success
was a test Windows XP machine with one hard drive.
We boot a Windows system from the CD, start up a P2V, it runs for an
apparently random length of time, then the P2V percentage just stops
somewhere. The other day was 13 percent. Today it's 8 percent.
Eventually, virt-p2v-server on the conversion server also stops and I
see a message in the log file that says, " virt-v2v: Error receiving
data:".
Is there anything we can look at from an alternate console on the source
server booted from that CD?
Thanks
- Greg
_______________________________________________
Libguestfs mailing list
Libguestfs(a)redhat.com
https://www.redhat.com/mailman/listinfo/libguestfs
7:26 p.m.
New subject: P2Vs seem to require a very robust Ethernet
I like it, we have the space. alternatively i build a physical fedora 16 box with 500 gb
of space... I can get on it tomorrow night
-----Original Message-----
From: Greg Scott [mailto:GregScott@Infrasupport.com]
Sent: Fri 12/2/2011 5:57 PM
To: Greg Scott; Matthew Booth; Richard W.M. Jones; libguestfs(a)redhat.com
Cc: Fredy Hernández; Joey Bertalan
Subject: RE: [Libguestfs] P2Vs seem to require a very robust Ethernet
After 5 or 6 hours, we aborted today's attempt, disconnected the 2nd HDD
(sdb) from the original host, and started up a new one.
Since all we have to go on are guesses - there was a bug with the
Windows virtio network driver where the driver would go offline during
bulk copies. I bring this up because our P2V conversion server is a
Fedora VM, probably using the Linux virtio driver. Was there a similar
bug with the Linux VirtIO driver?
One possible sequence of events:
- Data flies fast and furious in and out of the conversion server VM to
the NFS server.
- The conversion server VM NIC goes offline.
- The source host eventually hangs or dies.
- virt-p2v-server eventually gives up and writes a cryptic log message.
Another possible sequence of events:
- The source server starts copying.
- The source server runs into a problem and either hangs or dies.
- Virt-p2v-server on the conversion server eventually times out waiting
for data.
What do you think about trying a physical conversion server, possibly
using itself as the NFS Server and RHEV Export domain?
- Greg
-----Original Message-----
From: libguestfs-bounces(a)redhat.com
[mailto:libguestfs-bounces@redhat.com] On Behalf Of Greg Scott
Sent: Friday, December 02, 2011 3:18 PM
To: Matthew Booth; Richard W.M. Jones; libguestfs(a)redhat.com
Cc: Fredy Hernandez; Joey Bertalan
Subject: Re: [Libguestfs] P2Vs seem to require a very robust Ethernet
Our track record for P2Vs isn't very good so far; we've had one run to
completion in about 8 or 9 tries so far. We've tried source systems
with different versions of Windows, all but one failed. The one success
was a test Windows XP machine with one hard drive.
We boot a Windows system from the CD, start up a P2V, it runs for an
apparently random length of time, then the P2V percentage just stops
somewhere. The other day was 13 percent. Today it's 8 percent.
Eventually, virt-p2v-server on the conversion server also stops and I
see a message in the log file that says, " virt-v2v: Error receiving
data:".
Is there anything we can look at from an alternate console on the source
server booted from that CD?
Thanks
- Greg
_______________________________________________
Libguestfs mailing list
Libguestfs(a)redhat.com
https://www.redhat.com/mailman/listinfo/libguestfs
Fredy Hernandez
Systems Administrator
763-574-8734 Office
hernandez(a)stylmark.com
10:32 p.m.
New subject: P2Vs seem to require a very robust Ethernet
Here's another nugget of info. Watching top on my conversion server, I
consistently see roughly 50% id and 50% wa. It varies second by second
by a percent or two but it's generally around 50-50. The "wa" state
means waiting for an I/O to complete so it seems like something is
taking a **very** long time to complete IOs. My guess is, something is
stalling virt-p2v-server. I wonder if there's a way to drill down into
an individual process the way we used to do it with OpenVMS?
Every once-in-a-while, virt-p2v-server will jump up to the top of the
top list. The current P2V has been running for about 6 hours so far and
I don't know what the state looks like on the source server console.
The logfile on the conversion server is still blank. Watching the
Export domain with my RHEV Manager, I see this P2V has used about 28 GB
of space on the NFS server so far.
On the source server - how do we know the NIC speed and duplex? I guess
we can look at the switch port from the switch's point of view, but how
would we look at it and/or set it from the host point of view?
- Greg
6:24 a.m.
New subject: P2Vs seem to require a very robust Ethernet
14 hours after starting it up - about 41 GB done. The top utility still
shows about 50-50 between id and wa. Assuming virt-p2v-server is
stalling, I wonder how to find if it's stalling waiting on network I/O
from the source server or waiting on network I/O to the NFS server?
- Greg
6:40 a.m.
New subject: P2Vs seem to require a very robust Ethernet
And some more clues. Apparently, our virt-p2v-server is dropping lots
of received packets. I wonder why?
[root@Fedora16-64P2V log]# netstat -i
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP
TX-OVR Flg
eth0 1500 0 347387803 0 29625 0 134464820 0
0 0 BMRU
lo 16436 0 1088 0 0 0 1088 0 0
0 LRU
virbr0 1500 0 0 0 0 0 0 0 0
0 BMU
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]# netstat -i
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP
TX-OVR Flg
eth0 1500 0 347393913 0 29630 0 134465886 0
0 0 BMRU
lo 16436 0 1088 0 0 0 1088 0 0
0 LRU
virbr0 1500 0 0 0 0 0 0 0 0
0 BMU
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]# netstat -i
Kernel Interface table
Iface MTU Met RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP
TX-OVR Flg
eth0 1500 0 347394009 0 29635 0 134465895 0
0 0 BMRU
lo 16436 0 1088 0 0 0 1088 0 0
0 LRU
virbr0 1500 0 0 0 0 0 0 0 0
0 BMU
[root@Fedora16-64P2V log]#
6:57 a.m.
New subject: P2Vs seem to require a very robust Ethernet
I found a presentation from IBM that shows how to do a bunch of
drilldown stuff:
http://linuxvm.org/present/SHARE103/S9303ob.pdf
It mentions a utility named strace that shows system calls. I ran one
for a couple seconds against the running virt-p2v-server - here are the
results:
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]# ps ax | grep p2v
23339 ? Ds 2:44 /usr/bin/perl /usr/bin/virt-p2v-server
24048 pts/0 S+ 0:00 grep --color=auto p2v
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]# strace -p 23339 -tt -T -o greg.txt
Process 23339 attached - interrupt to quit
^CProcess 23339 detached
[root@Fedora16-64P2V log]# more greg.txt
06:53:04.258833 write(5, "R\0e\0q\0u\0e\0s\0t\0:\0 \0N\0u\0l\0l\0;\0 \0
\0"..., 8192) = 8
192 <0.000107>
06:53:04.259246 write(5, "I\0n\0s\0t\0a\0l\0l\0e\0d\0:\0
\0N\0u\0l\0l\0;\0"..., 8192) = 8
192 <0.000100>
06:53:04.259503 write(5,
"e\0.\0x\0m\0l\0_\0004\0005\0003\0009\0004\0_\0e\0n\0u\0_\0"...,
8192) = 8192 <0.000126>
06:53:04.259821 write(5, "m\0p\0o\0n\0e\0n\0t\0:\0
\0_\0_\0V\0W\0D\0_\0W\0"..., 8192) = 8
192 <0.227877>
06:53:04.487853 write(5, "n\0t\0:\0
\0_\0_\0_\0S\0Q\0L\0D\0B\0G\0_\0004\0008\0"..., 8192)
= 8192 <0.000105>
06:53:04.488107 write(5,
"P\0L\0A\0T\0E\0S\0_\0x\0008\0006\0_\0e\0n\0u\0.\0003\0"..., 819
2) = 8192 <0.000101>
06:53:04.488357 write(5,
"_\0A\0005\0003\0006\0_\0000\0000\0009\0000\0002\0007\0008\0A\00
01\0B\0"..., 8192) = 8192 <0.000101>
06:53:04.488664 write(5,
"3\0002\0_\0_\0_\0_\0_\0x\0008\0006\0.\0003\0006\0004\0003\0002\
0"..., 8192) = 8192 <0.228029>
06:53:04.716874 write(5,
"A\0005\0003\0006\0_\0000\0000\0009\0000\0002\0007\0008\0A\0001\
0B\0B\0"..., 8192) = 8192 <0.000112>
06:53:04.717150 write(5, " \0 \0 \0A\0c\0t\0i\0o\0n\0:\0
\0N\0u\0l\0l\0\r\0"..., 8192) =
8192 <0.000108>
06:53:04.717409 write(5, "\r\0\n\0M\0S\0I\0 \0(\0s\0)\0
\0(\0004\0004\0:\0F\0000\0"..., 8
192) = 8192 <0.000112>
06:53:04.717683 write(5, " \0(\0s\0)\0 \0(\0004\0004\0:\0F\0000\0)\0
\0[\0002\0002\0"...,
8192) = 8192 <0.227951>
06:53:04.945788 write(5, "S\0I\0 \0(\0s\0)\0
\0(\0004\0004\0:\0F\0000\0)\0 \0[\0"..., 819
2) = 8192 <0.000105>
06:53:04.946041 write(5, ")\0 \0(\0004\0004\0:\0F\0000\0)\0
\0[\0002\0002\0:\0003\0000\0"
..., 8192) = 8192 <0.000100>
06:53:04.946288 write(5,
".\0003\0006\0004\0003\0002\0003\0006\0F\0_\0F\0C\0007\0000\0_\0
001\0"..., 8192) = 8192 <0.000104>
06:53:04.946540 write(5,
"n\0g\0_\0_\0_\0V\0S\0D\0e\0b\0u\0g\0_\0R\0e\0g\0"..., 8192) = 8
192 <0.229116>
06:53:05.175810 write(5, "I\0n\0s\0t\0a\0l\0l\0e\0d\0:\0
\0N\0u\0l\0l\0;\0"..., 8192) = 8
192 <0.000105>
06:53:05.176062 write(5,
"_\0M\0i\0c\0r\0o\0s\0o\0f\0t\0_\0V\0C\0009\0000\0_\0"..., 8192)
= 8192 <0.000109>
06:53:05.176319 write(5,
"s\0_\0R\0e\0s\0_\0r\0u\0s\0_\0004\0002\0005\0003\0.\0003\0"...,
8192) = 8192 <0.000120>
06:53:05.176593 write(5, "l\0e\0d\0:\0 \0N\0u\0l\0l\0;\0 \0 \0
\0R\0e\0q\0"..., 8192) = 8
192 <0.228033>
06:53:05.404780 write(5, "u\0l\0l\0\r\0\n\0M\0S\0I\0 \0(\0s\0)\0
\0(\0004\0004\0"..., 819
2) = 8192 <0.000106>
06:53:05.405033 write(5, "l\0;\0 \0 \0 \0R\0e\0q\0u\0e\0s\0t\0:\0
\0N\0u\0"..., 8192) = 8
192 <0.000101>
06:53:05.405280 write(5,
"5\0009\0000\0000\0_\0e\0n\0u\0_\0x\0008\0006\0.\0003\0006\0004\
0"..., 8192) = 8192 <0.000104>
06:53:05.405641 write(5,
"b\0_\0002\0003\0002\0003\0000\0.\0003\0006\0004\0003\0002\0003\
0006\0F\0"..., 8192) = 8192 <0.230937>
06:53:05.636740 write(5, "e\0s\0t\0:\0 \0L\0o\0c\0a\0l\0;\0 \0 \0
\0A\0c\0"..., 8192) = 8
192 <0.000106>
06:53:05.636996 write(5, "o\0n\0:\0 \0N\0u\0l\0l\0\r\0\n\0M\0S\0I\0
\0(\0s\0"..., 8192) =
8192 <0.000106>
06:53:05.637250 write(5, "t\0:\0
\0_\0_\0K\0B\0009\0005\0004\0003\0004\0005\0.\0I\0n\0"..
., 8192) = 8192 <0.000104>
06:53:05.637512 write(5,
"A\0005\0003\0006\0_\0000\0000\0009\0000\0002\0007\0008\0A\0001\
0B\0B\0"..., 8192) = 8192 <0.233094>
06:53:05.870776 write(5,
"R\0i\0b\0b\0o\0n\0_\0C\0a\0t\0e\0g\0o\0r\0y\0C\0"..., 8192) = 8
192 <0.000108>
06:53:05.871042 write(5, "S\0I\0 \0(\0s\0)\0
\0(\0004\0004\0:\0F\0000\0)\0 \0[\0"..., 819
2) = 8192 <0.000101>
06:53:05.871296 write(5,
"_\0I\0n\0c\0l\0u\0d\0e\0_\0B\0l\0a\0c\0k\0_\0M\0"..., 8192) = 8
192 <0.000106>
06:53:05.871609 write(5, "(\0s\0)\0 \0(\0004\0004\0:\0F\0000\0)\0
\0[\0002\0002\0:\0"...,
8192) = 8192 <0.228984>
06:53:06.100767 write(5, "e\0d\0:\0 \0A\0b\0s\0e\0n\0t\0;\0 \0 \0
\0R\0e\0"..., 8192) = 8
192 <0.000109>
06:53:06.101038 write(5, "l\0e\0d\0:\0 \0L\0o\0c\0a\0l\0;\0 \0 \0
\0R\0e\0"..., 8192) = 8
192 <0.000105>
06:53:06.101293 write(5, "a\0l\0;\0 \0 \0 \0A\0c\0t\0i\0o\0n\0:\0
\0N\0u\0"..., 8192) = 8
192 <0.000103>
06:53:06.101553 write(5, "n\0t\0;\0 \0 \0 \0R\0e\0q\0u\0e\0s\0t\0:\0
\0L\0"..., 8192 <unf
inished ...>
[root@Fedora16-64P2V log]#
7:03 a.m.
New subject: P2Vs seem to require a very robust Ethernet
Trying strace again didn't work very well - I wonder if it was stuck
waiting for a network I/O to finish?
[root@Fedora16-64P2V log]# ps ax | grep p2v
23339 ? Ds 2:44 /usr/bin/perl /usr/bin/virt-p2v-server
24048 pts/0 S+ 0:00 grep --color=auto p2v
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]# strace -p 23339 -tt -T
Process 23339 attached - interrupt to quit
^C^C^C^C^C^C^C^C^C^C^C^C
^Z
[1]+ Stopped strace -p 23339 -tt -T
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]# ps ax | grep p2v
23339 ? Ds 2:45 /usr/bin/perl /usr/bin/virt-p2v-server
24054 pts/0 R+ 0:00 grep --color=auto p2v
[root@Fedora16-64P2V log]# ps ax | grep strace
24052 pts/0 T 0:00 strace -p 23339 -tt -T
24056 pts/0 S+ 0:00 grep --color=auto strace
[root@Fedora16-64P2V log]# attach -h
bash: attach: command not found...
[root@Fedora16-64P2V log]# kill -9 24052
[1]+ Killed strace -p 23339 -tt -T
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]# ps ax | grep p2v
23339 ? Ds 2:45 /usr/bin/perl /usr/bin/virt-p2v-server
24065 pts/0 S+ 0:00 grep --color=auto p2v
[root@Fedora16-64P2V log]#
7:13 a.m.
New subject: P2Vs seem to require a very robust Ethernet
Trying strace against virt-p2v-server again, this time to stdout -
A bunch of writes, then another hang. I annotated this one a little bit. That hang went
for more than 3 **minutes** from one write to the next. Combined with a top display
showing the overall system is spending about 1/2 its time waiting for I/Os to complete,
does this tell us anything?
.
.
.
07:04:10.643462 write(5,
"\0\0\0\0\20\220\362\10\0\0\0\377\376\377R\\\0\\\0P\0e\0t\0a\0g\0o\0n"..., 8192)
= 8192 <0.227277>
07:04:10.870894 write(5,
"\1\0\0\0\0\0Iw\330@\377\376\377\10M\0o\0d\0i\0f\0i\0e\0d\0\5\0"..., 8192) =
8192 <0.000032>
07:04:10.871052 write(5,
"\0\7\201\377\376\377\0062\0001\0001\0001\0002\0003\0\2\0\0\210\360\2617\0G\4\4K\6"...,
8192) = 8192 <0.000035>
07:04:10.871211 write(5,
"-\vd\216^\7\10\266\274\330\214}N\2\2xN\263i,\271\5\242\242w\374v\357\315\233O\34"...,
8192) = 8192 <0.000033>
07:04:10.871423 write(5,
"\377\377\377\377\377\377\377\377L\377\0\0\377\377\377\377\377\377\377\377\377\377L\377\0\0\377\377\377\377\377\377"...,
8192) = 8192 <0.227303>
07:04:11.098886 write(5,
"\2\0\0\5\7\t\366\377\366\0\2\0\4\t\2\0\377\377\377\377\377\377\24\377\0\0\377\377\355\377\2\0"...,
8192) = 8192 <0.000034>
07:04:11.099092 write(5,
"\1\0\377\377\345\377\0\0\377\377\377\377\377\377\34\377\2\0\31\7\3\0\26\7\10\0\2\366\0\4\0\7"...,
8192) = 8192 <0.000031>
07:04:11.099294 write(5,
"\377\377L\377\0\0\377\377\377\377\377\377\377\377\377\377L\377\0\0\377\377\377\377\377\377\377\377\377\377L\377"...,
8192) = 8192 <0.000036>
07:04:11.099503 write(5,
"N\0D\0_\0P\0I\0N\0S\0\20\0\0\0\0\0\0\0\0\0\1\0\0\0\1\0\0\0"..., 8192
(I inserted blank space here - note the time indexes - more than 3 minutes when nothing
happened.)
) = 8192 <195.972190>
07:07:27.071878 write(5,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 8192) =
8192 <0.000100>
07:07:27.072111 write(5,
"\35\202\0\0\364\201\305\2\247\1\0\0\240\336\2469\362\201\0\0\364\201\305\2\307\1\0\0\365R\2659"...,
8192) = 8192 <0.000046>
07:07:27.072269 write(5,
"\5\0\0\0\35\202\0\0\364\201\366\1\247\1\0\0\240\336\2469\362\201\0\0\364\201\366\1\307\1\0\0"...,
8192) = 8192 <0.000049>
07:07:27.072420 write(5,
"\364\201\366\1\247\1\0\0\240\336\2469\362\201\0\0\364\201\366\1\307\1\0\0\365R\2659\2\0\0\0"...,
8192) = 8192 <0.226265>
07:07:27.298847 write(5,
"?\32p\303u\371\234\355?\2\352%S\253\374\354?\314\314\235\373&#\257\2770u.\301L\21\202"...,
8192) = 8192 <0.000106>
07:07:27.299069 write(5,
"\357\345\313\303\260\277\177\23\256\256B\370\376]\235_3\263\312U\t4'
\0006g\353\312\4\10$"..., 8192) = 8192 <0.000100>
07:07:27.299269 write(5, "@ 1\1\0010\261\1\327]\2\4\10\20 @\200\200\0h\16\20
@\200\0\1\2\4\22\23\20"..., 8192) = 8192 <0.000100>
07:07:27.299470 write(5,
"F%u\2\212?\1\0\0\0\0\0\0\0\377\376\377\0\377\376\377\0\0\0\0\0\0\0\0\0\0\0"...,
8192) = 8192 <0.227184>
07:07:27.526791 write(5,
"\303&\234\334\314\345?\0\0\0\0\0\0\0\0\347\260x\307\207\324\356?\3407\200et%\321?\330"...,
8192) = 8192 <0.000051>
07:07:27.526952 write(5,
"\6\2K\0\0\0x\235pI\5\0\0\0\377\377\377\377\0\0\0\0\3\202\0\0\364\201\6\2\366\0"...,
8192) = 8192 <0.000052>
07:07:27.527156 write(5,
"\300\300\240\0\360\373\377\0\244\240\240\0\200\200\200\0\0\0\377\0\0\377\0\0\0\377\377\0\377\0\0\0"...,
8192) = 8192 <0.000057>
07:07:27.527361 write(5,
"}\377\2\0T\7\2\0\1\7\1\0\2\7\5\0\4\366\2\0\2\7\1\10\2\366\3\377\1\t\2\0"...,
8192^C <unfinished ...>
Process 23339 detached
[root@Fedora16-64P2V log]#
11:46 a.m.
New subject: P2Vs seem to require a very robust Ethernet
Watching strace -p 23339 -tt -T in another window, where 23339 is the PID of our
virt-v2v-server process - reads just machine gun past. But now it's doing a bunch of
writes and they blip past kind of like individual shotgun blasts. OK, lousy metaphor but
you get the idea. And some of the writes hang for a while, like the one I caught earlier.
In fact, I just grabbed another couple of hanging writes - see below. I inserted a bunch
of blank space at each hang. And now that we have some footprints, see an action plan
below that.
.
.
.
10:58:04.647333 write(5,
"*\215N\367\273\377\1\0\0\323\343\v\323\213\312\301\371\10\210\17GN\301\342\10\203\376\v~2\203\356"...,
8192) = 8192 <0.233197>
10:58:04.880702 write(5,
"\310\323\343\3\323\213\336\203\300\10\353\10\205\300\17\214\3\2\0\0\212\2174\354\2\0\204\311\2134\275X"...,
8192) = 8192 <0.000043>
10:58:04.880935 write(5,
"\306\213\3513\3523\357\301\311\2\3\305\213t$`\213l$h3\365\213\254$\200\0\0\0003\365\213"...,
8192) = 8192 <0.000034>
10:58:04.881103 write(5,
"\n\213\302\203\360\377\3\373\213i8\v\3073\303\3\365\5\247#\224\253\3\360\301\306\17\213\303\203\360\377"...,
8192) = 8192 <0.000033>
10:58:04.881269 write(5,
"\311\302\34\0\314\314\314\314\314\314\213\377U\213\354\213M\20\213E\fS3\333\210\10@C\205\311\17\216"...,
8192) = 8192 <0.233241>
10:58:05.114669 write(5,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 8192) =
8192 <0.000038>
10:58:05.114932 write(5, "WithTag\0N\0ExFreePoolWithTag\0\265\5st"..., 8192) =
8192 <0.000035>
10:58:05.115095 write(5,
".rdata\0\0$\2\0\0\200C\0\0\200\2\0\0\200C\0\0\0\0\0\0\0\0\0\0"..., 8192) = 8192
<0.000039>
10:58:05.115306 write(5,
"\377\377\351\263\376\377\377j\0j\1\215\206\264\2\0\0P\377\25\10D\1\0\351\254\376\377\377\215\216\234"...,
8192) = 8192 <0.233190>
10:58:05.348659 write(5,
"\2\2111^[_]\302(\0\314\314\314\314\314\314\377%\264C\1\0\314\314\314\314\314\314\377%\270C"...,
8192) = 8192 <0.000038>
10:58:05.348884 write(5,
"\356\1\0\0\0\25\0\0\0\2\0\0\0\25\0\0\0\0\0\0\0\0\0\0\0\0\0\0@\0\0H"..., 8192) =
8192 <0.000035>
10:58:05.349055 write(5, "P\0\0\0\32\0\0\0\0\5\0\0
\3\0\0\0\17\0\0\1\0\0\0\30\0\0\0<\0\0\0"..., 8192) = 8192 <0.000082>
10:58:05.349311 write(5,
"u\30\377u\24\377u\20S\377u\10\350\375\30\1\0;\307\211E$\17\204\204\2\0\0009}\364\17"...,
8192
) = 8192 <63.979352>
10:59:09.328867 write(5,
"\1\0\0\213K\4\213E\350;H\ft\r\213M\10\366A\34\1\17\205L\1\0\0\377u\10P\350"...,
8192) = 8192 <0.000070>
10:59:09.329110 write(5,
"WP\350s\331\0\0\211\2050\377\377\377\213M\374\213\2050\377\377\377_^[\350\333\324\0\0\311\302"...,
8192) = 8192 <0.000048>
10:59:09.329268 write(5,
"\354\f\213\r\250Q\367\277SV\301\341\5\213\221\300Q\367\277\213\201\310Q\367\277W\213}\10\213w$"...,
8192) = 8192 <0.000050>
10:59:09.329422 write(5,
"\17\204\244\0\0\0\213G\4;C\fu\31\377u\20\215E\350P\350\354p\377\377j\0S\350\212\234"...,
8192) = 8192 <0.536185>
10:59:09.865891 write(5,
"\300\213E\274\3\310\211M\334\213\n;\371}\2\213\317\3\310\213B\3749E\304\211M\344\213M\304\177"...,
8192) = 8192 <0.000169>
10:59:09.866274 write(5,
"\277\211\204\21\354\27\1\0\2418@\367\277\203\274\1\354\27\1\0\0\17\204\362\0\0\0\213O\f#\313"...,
8192) = 8192 <0.000163>
10:59:09.866663 write(5,
"\300\212g\367\212G\370f\205\300t\3\200\n\0043\300\212g\371\212G\372f\205\300t\3\200\n\0103"...,
8192) = 8192 <0.000168>
10:59:09.867043 write(5,
"\0w\n\306\2\320\212\331B\376\313\353\351\306\2\367Bf\211\nBB\213]\364\210Z\3\301\353\10\210"...,
8192
) = 8192 <189.274151>
11:02:19.141412 write(5,
"\0\4\0\4\2\6\2\6\0\4\0\4\2\6\2\6\1\5\1\5\3\7\3\7\1\5\1\5\3\7\3\7"..., 8192) =
8192 <0.233716>
11:02:19.375339 write(5,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 8192) =
8192 <0.000288>
11:02:19.375800 write(5, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0.text\0\0\0
4\t\0\0\20\0\0"..., 8192) = 8192 <0.000269>
11:02:19.376243 write(5,
"L\2\0\0\205\300t\23Pj\0\377\25|V\t\20\307\206L\2\0\0\0\0\0\0\213\206P\2\0"...,
8192) = 8192 <0.000279>
11:02:19.376693 write(5, "\0\0\302\10\0j\0S\215L$
Q\213\317\350\234\365\377\377\213\360\205\366|\305\213\317\350\317\345\377"..., 8192)
= 8192 <0.234436>
11:02:19.611311 write(5,
"U\213\354j\377h\373\332\10\20d\241\0\0\0\0Pd\211%\0\0\0\0\203\354\24S\213]\20V"...,
8192) = 8192 <0.000288>
11:02:19.611772 write(5,
"\3\0\0\213\317\306D$\34\4\350i\237\7\0\307\7\24\371\t\20\215\276h\3\0\0\213\317\306D$"...,
8192) = 8192 <0.000295>
11:02:19.612241 write(5,
"\374\353\0023\300\213P,\271\1\0\0\0;\321t\r\211H,\213@\10\213\20QP\377R\f_3"...,
8192) = 8192 <0.000265>
11:02:19.612677 write(5,
"\213\6W\213\316\307F,\270\2\0\0\377P8\215L$\20Q3\333h\314\263\t\20\211\\$\30\377"...,
8192) = 8192 <0.232419>
11:02:19.845273 write(5,
"V\213\361\213Fd\205\300t&\213F`\205\300t\7P\377\25\314V\t\20\213FhP\377\25\330R"...,
8192) = 8192 <0.000290>
.
.
.
Based on the evidence we've been able to uncover, it feels to me like we're having
a problem writing to the NFS server. Here's why I think so.
virt-p2v-server pretty much reads blocks from the source server and writes to the Export
NFS share - right? Reads are fast and furious, writes are very slow and sometimes even
hang for ***minutes*** at a time.
If my hunch is right, then it seems to me we either have an issue with the virtual NIC on
our Fedora16-64P2V VM, or the NFS server, or the network in-between.
The source server isn't part of the picture because we've tried several source
servers and the behavior is the same for every one.
We've simplified the network as much as we can. We have a brand new HP Procurve
2910al-24G and all the players are directly connected to that switch. So I think we can
eliminate the network since all the transactions travel over the same switch, just to/from
different sources and destinations.
That leaves either the conversion server or NFS server as culprits.
So here's what I'd like to try:
Let's eliminate the Storagetek NFS Server first.
Let's build up that 500 GB box Fredy mentioned as a physical F16 system and set it up
as an NFS server. Change the RHEV Export domain to point to this new NFS server and try
some P2Vs using the same Fedora VM as our conversion server. Observe the results with
strace and overall.
If this doesn't change the symptoms, then let's add the virt-v2v stuff to that 500
GB Fedora system and we'll use it both as a conversion server and NFS server and try
again.
And finally, to round out the testing, if we still have energy left, we can set up the
Storagetek NFS server again as the RHEV export domain and try some more P2Vs with that 500
GB physical Fedora system to the Storagetek.
- Greg
6:56 a.m.
New subject: P2Vs seem to require a very robust Ethernet
40+ hour after starting, that P2V is **still** running and the logfile is 0 length. But
curiously, the RHEV Export storage it wants has not increased in about the past 12+ hours.
The top display is now showing 99%+ idle. Could virt-p2v-server be stuck in a loop of
some kind?
- Greg
7:08 a.m.
New subject: P2Vs seem to require a very robust Ethernet
...And a few seconds after clicking "Send" it finally went up by one more GB.
As of right now, we're at 89 total GB used in the RHEV Exports storage. The
ENG-BACKUP we did a while ago uses 38 GB, and there's some overhead, so this P2V must
be around 45 - 50+ GB used so far.
- Greg
-----Original Message-----
From: libguestfs-bounces(a)redhat.com [mailto:libguestfs-bounces@redhat.com] On Behalf Of
Greg Scott
Sent: Sunday, December 04, 2011 6:57 AM
To: Matthew Booth; Richard W.M. Jones; libguestfs(a)redhat.com
Cc: Joey Bertalan
Subject: Re: [Libguestfs] P2Vs seem to require a very robust Ethernet
40+ hour after starting, that P2V is **still** running and the logfile is 0 length. But
curiously, the RHEV Export storage it wants has not increased in about the past 12+ hours.
The top display is now showing 99%+ idle. Could virt-p2v-server be stuck in a loop of
some kind?
- Greg
_______________________________________________
Libguestfs mailing list
Libguestfs(a)redhat.com
https://www.redhat.com/mailman/listinfo/libguestfs
7:35 p.m.
New subject: P2Vs seem to require a very robust Ethernet
Well now this is very strange. That P2V we started up Friday afternoon crossed into hour
52 a few minutes ago. But in the past 3 1/2 hours, it hasn't used one more byte of
disk space in the NFS Export service. But then after firing up an strace just now, it
started using space again. What's up with that?
[root@Fedora16-64P2V log]# date
Sun Dec 4 15:52:22 CST 2011
[root@Fedora16-64P2V log]# df
Filesystem 1K-blocks Used Available Use% Mounted on
rootfs 49058108 3944064 42622040 9% /
devtmpfs 2014608 0 2014608 0% /dev
tmpfs 2024924 444 2024480 1% /dev/shm
tmpfs 2024924 48480 1976444 3% /run
/dev/mapper/vg_fedora1664p2v-lv_root 49058108 3944064 42622040 9% /
tmpfs 2024924 48480 1976444 3% /run
tmpfs 2024924 0 2024924 0% /sys/fs/cgroup
tmpfs 2024924 0 2024924 0% /media
/dev/vda2 495844 82656 387588 18% /boot
175.10.0.45:/EXPORTS/ 1055113408 93131296 961982112 9% /tmp/UBKmK8QbfM
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]# date
Sun Dec 4 19:25:52 CST 2011
[root@Fedora16-64P2V log]# df
Filesystem 1K-blocks Used Available Use% Mounted on
rootfs 49058108 3944084 42622020 9% /
devtmpfs 2014608 0 2014608 0% /dev
tmpfs 2024924 444 2024480 1% /dev/shm
tmpfs 2024924 48480 1976444 3% /run
/dev/mapper/vg_fedora1664p2v-lv_root 49058108 3944084 42622020 9% /
tmpfs 2024924 48480 1976444 3% /run
tmpfs 2024924 0 2024924 0% /sys/fs/cgroup
tmpfs 2024924 0 2024924 0% /media
/dev/vda2 495844 82656 387588 18% /boot
175.10.0.45:/EXPORTS/ 1055113408 93131296 961982112 9% /tmp/UBKmK8QbfM
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]# ps ax | grep p2v
23339 ? Ts 3:13 /usr/bin/perl /usr/bin/virt-p2v-server
25668 pts/0 S+ 0:00 grep --color=auto p2v
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]# strace -p 23339 -tt -T
.
.
.
19:27:20.968661 write(5, "a\22-\7
\2\1\16\21\211\305\v\7\4\22\202\361\22\202\365\35\16\10\5 \0\22\211\315\6 \1"...,
8192) = 8192 <0.000029>
19:27:20.968921 write(5, "\22-\6
\2\1\35\16\10\r\7\6\22\201e\16\16\35\16\22a\22-\26\7\f\16\22\210 \35\16"..., 8192) =
8192 <0.000027>
19:27:20.969187 write(5,
"\16\n\7\4\22\203\261\10\22a\22-#\7\t\22\203\234\22\207\360\10\10\22\207\350\25\22\201\225\1\22"...,
8192) = 8192 <0.000026>
19:27:20.969441 write(5, "\0\0\36\1\0\31MasterPage_MasterPageFile\0"..., 8192) =
8192 <0.000114>
19:27:20.969917 write(5,
"\16\16\34\22\224\10\22\204-\22]\22-\22]\35\34\22-\23\7\10\22\203\235\16\22\201i\2\22\204"...,
8192^C <unfinished ...>
Process 23339 detached
[root@Fedora16-64P2V log]# date
Sun Dec 4 19:31:35 CST 2011
[root@Fedora16-64P2V log]# df
Filesystem 1K-blocks Used Available Use% Mounted on
rootfs 49058108 3944084 42622020 9% /
devtmpfs 2014608 0 2014608 0% /dev
tmpfs 2024924 444 2024480 1% /dev/shm
tmpfs 2024924 48480 1976444 3% /run
/dev/mapper/vg_fedora1664p2v-lv_root 49058108 3944084 42622020 9% /
tmpfs 2024924 48480 1976444 3% /run
tmpfs 2024924 0 2024924 0% /sys/fs/cgroup
tmpfs 2024924 0 2024924 0% /media
/dev/vda2 495844 82656 387588 18% /boot
175.10.0.45:/EXPORTS/ 1055113408 93135072 961978336 9% /tmp/UBKmK8QbfM
[root@Fedora16-64P2V log]#
7:48 p.m.
New subject: P2Vs seem to require a very robust Ethernet
Our P2V conversion server is a RHEV VM. I wonder if there are BIOS settings on the hosts
about going into standby?
Well now this is very strange. That P2V we started up Friday
afternoon
crossed into hour 52 a few minutes ago. But in the past 3 1/2 hours, it
hasn't used one more byte of disk space in the NFS Export service. But
then after firing up an strace just now, it started using space again.
What's up with that?
- Greg
3:24 a.m.
New subject: P2Vs seem to require a very robust Ethernet
This time after roughly 7 1/2 hours, the exact amount of disk space used in the NFS Export
domain. And then as soon as I run df a couple of times, now the disk space usage starts
to climb again. One other symptom to note - that 3rd df took nearly 2 minutes to respond.
See the date commands before and after. The first 2 df commands responded immediately.
After the 2nd df command, I looked at virt-p2v-server with strace again.
I don't think my RHEV hosts are going into standby. If they were, then why would my
df and strace commands to a guest VM on one of the hosts respond immediately? I think the
Storagetek NFS server goes offline and somehow when I either do a df command or strace
against virt-p2v-server, it goes back online again.
We'll know more when we try this same thing with a different NFS server hopefully
after the sun comes up today.
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]# date
Sun Dec 4 19:49:06 CST 2011
[root@Fedora16-64P2V log]# df
Filesystem 1K-blocks Used Available Use% Mounted on
rootfs 49058108 3944084 42622020 9% /
devtmpfs 2014608 0 2014608 0% /dev
tmpfs 2024924 444 2024480 1% /dev/shm
tmpfs 2024924 48480 1976444 3% /run
/dev/mapper/vg_fedora1664p2v-lv_root 49058108 3944084 42622020 9% /
tmpfs 2024924 48480 1976444 3% /run
tmpfs 2024924 0 2024924 0% /sys/fs/cgroup
tmpfs 2024924 0 2024924 0% /media
/dev/vda2 495844 82656 387588 18% /boot
175.10.0.45:/EXPORTS/ 1055113408 93212416 961900992 9% /tmp/UBKmK8QbfM
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]#
[root@Fedora16-64P2V log]# date
Mon Dec 5 03:10:49 CST 2011
[root@Fedora16-64P2V log]# df
Filesystem 1K-blocks Used Available Use% Mounted on
rootfs 49058108 3944144 42621960 9% /
devtmpfs 2014608 0 2014608 0% /dev
tmpfs 2024924 444 2024480 1% /dev/shm
tmpfs 2024924 48476 1976448 3% /run
/dev/mapper/vg_fedora1664p2v-lv_root 49058108 3944144 42621960 9% /
tmpfs 2024924 48476 1976448 3% /run
tmpfs 2024924 0 2024924 0% /sys/fs/cgroup
tmpfs 2024924 0 2024924 0% /media
/dev/vda2 495844 82656 387588 18% /boot
175.10.0.45:/EXPORTS/ 1055113408 93212416 961900992 9% /tmp/UBKmK8QbfM
[root@Fedora16-64P2V log]# date
Mon Dec 5 03:12:37 CST 2011
[root@Fedora16-64P2V log]# df
Filesystem 1K-blocks Used Available Use% Mounted on
rootfs 49058108 3944144 42621960 9% /
devtmpfs 2014608 0 2014608 0% /dev
tmpfs 2024924 444 2024480 1% /dev/shm
tmpfs 2024924 48476 1976448 3% /run
/dev/mapper/vg_fedora1664p2v-lv_root 49058108 3944144 42621960 9% /
tmpfs 2024924 48476 1976448 3% /run
tmpfs 2024924 0 2024924 0% /sys/fs/cgroup
tmpfs 2024924 0 2024924 0% /media
/dev/vda2 495844 82656 387588 18% /boot
175.10.0.45:/EXPORTS/ 1055113408 93266752 961846656 9% /tmp/UBKmK8QbfM
[root@Fedora16-64P2V log]# date
Mon Dec 5 03:14:18 CST 2011
[root@Fedora16-64P2V log]#
- Greg
3:11 p.m.
New subject: P2Vs seem to require a very robust Ethernet
Very strange behavior. Fredy killed the P2V at the source server console about 3 hours
ago. After roughly 67 hours, the bar graph on the console was at 60 percent. I just
checked and noticed virt-p2v-server was still running on the conversion server. So I did
an strace in a different window. Tons of output immediately poured out and
virt-p2v-server ended. Below is what I still have in the putty window's buffer.
I'll bet it was stalled writing to the NFS Export share. Why doing an strace makes it
come alive again is a mystery. Or a miracle, depending on your point of view I guess.
- Greg
14:28:44.438657 read(0,
"\0\1\0\0\0\0\0\0\177\0\3\0z\0314\0Z;;\0;\4n\0K\6Z\0v\37C\0"..., 8192) = 8192
<0.000033>
14:28:44.439021 read(0,
"\0\0\0002\0\0\0\22\0\0\0\10\0\0\0\1\0\0\0\0\0\0\0\27N\0\1\313\324i9\377"...,
8192) = 8192 <0.000033>
14:28:44.439357 read(0,
"\364\265\202\377\367\271\205\377\370\262u\377\371\254f\377\373\252\\\377\374\263b\377\365\275o\377z)\31\337"...,
8192) = 8192 <0.000032>
14:28:44.439686 read(0,
"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\10ww"..., 8192) =
8192 <0.000032>
14:28:44.440057 read(0, "", 8192) = 0 <0.000028>
14:28:44.440390 mremap(0x7fe85e187000, 4198400, 8294400, MREMAP_MAYMOVE) = 0x7fe85cd9b000
<0.000931>
14:28:44.458613 write(1, "ERROR Error receiving data: \n\n", 30) = -1 EPIPE
(Broken pipe) <0.000139>
14:28:44.459051 --- {si_signo=SIGPIPE, si_code=SI_USER, si_pid=23339, si_uid=0} (Broken
pipe) ---
14:28:44.459565 sendto(3, "<11>Dec 5 14:28:44 virt-v2v[233"..., 67,
MSG_NOSIGNAL, NULL, 0) = 67 <0.001088>
14:28:44.460994 write(2, "virt-v2v: Error receiving data: "..., 34) = 34
<0.000064>
14:28:44.461376 rt_sigaction(SIG_0, NULL, {0x34772bbf32, ~[RT_1 RT_2 RT_3 RT_4 RT_5 RT_6
RT_7 RT_8 RT_9 RT_10 RT_11 RT_12 RT_13 RT_14 RT_15 RT_16 RT_17 RT_18 RT_19 RT_20 RT_21
RT_22 RT_23 RT_24 RT_25 RT_26 RT_27 RT_28 RT_29 RT_30 RT_31], 0}, 8) = -1 EINVAL (Invalid
argument) <0.000029>
14:28:44.461746 rt_sigaction(SIGHUP, NULL, {SIG_DFL, [], SA_RESTORER, 0x347620f4f0}, 8) =
0 <0.000028>
14:28:44.462108 rt_sigaction(SIGINT, NULL, {SIG_DFL, [], SA_RESTORER, 0x347620f4f0}, 8) =
0 <0.000028>
14:28:44.462420 rt_sigaction(SIGQUIT, NULL, {SIG_DFL, [], SA_RESTORER, 0x347620f4f0}, 8) =
0 <0.000028>
14:28:44.462728 rt_sigaction(SIGILL, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000025>
14:28:44.463081 rt_sigaction(SIGTRAP, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000026>
14:28:44.463389 rt_sigaction(SIGABRT, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000026>
14:28:44.463695 rt_sigaction(SIGBUS, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000026>
14:28:44.464062 rt_sigaction(SIGFPE, NULL, {SIG_IGN, [FPE], SA_RESTORER|SA_RESTART,
0x3475e36300}, 8) = 0 <0.000025>
14:28:44.464374 rt_sigaction(SIGKILL, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000026>
14:28:44.464680 rt_sigaction(SIGUSR1, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000025>
14:28:44.465027 rt_sigaction(SIGSEGV, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000084>
14:28:44.465483 rt_sigaction(SIGUSR2, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000036>
14:28:44.465865 rt_sigaction(SIGPIPE, NULL, {SIG_IGN, [], SA_RESTORER, 0x347620f4f0}, 8) =
0 <0.000028>
14:28:44.466275 rt_sigaction(SIGALRM, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000027>
14:28:44.466648 rt_sigaction(SIGTERM, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000027>
14:28:44.467042 rt_sigaction(SIGSTKFLT, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000033>
14:28:44.467401 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000026>
14:28:44.467712 rt_sigaction(SIGCONT, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000025>
14:28:44.468088 rt_sigaction(SIGSTOP, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000025>
14:28:44.468397 rt_sigaction(SIGTSTP, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000025>
14:28:44.468703 rt_sigaction(SIGTTIN, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000025>
14:28:44.469059 rt_sigaction(SIGTTOU, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000025>
14:28:44.469368 rt_sigaction(SIGURG, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000026>
14:28:44.469674 rt_sigaction(SIGXCPU, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000025>
14:28:44.470022 rt_sigaction(SIGXFSZ, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000025>
14:28:44.470333 rt_sigaction(SIGVTALRM, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000025>
14:28:44.470637 rt_sigaction(SIGPROF, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000026>
14:28:44.470975 rt_sigaction(SIGWINCH, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000026>
14:28:44.471293 rt_sigaction(SIGIO, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000025>
14:28:44.471599 rt_sigaction(SIGPWR, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000025>
14:28:44.471904 rt_sigaction(SIGSYS, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000026>
14:28:44.472272 rt_sigaction(SIGRT_2, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000025>
14:28:44.472580 rt_sigaction(SIGRT_3, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000025>
14:28:44.472887 rt_sigaction(SIGRT_4, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000095>
14:28:44.473287 rt_sigaction(SIGRT_5, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000079>
14:28:44.473594 rt_sigaction(SIGRT_6, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000076>
14:28:44.473883 rt_sigaction(SIGRT_7, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000079>
14:28:44.474207 rt_sigaction(SIGRT_8, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000055>
14:28:44.474480 rt_sigaction(SIGRT_9, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000053>
14:28:44.474753 rt_sigaction(SIGRT_10, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000053>
14:28:44.475071 rt_sigaction(SIGRT_11, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000054>
14:28:44.475350 rt_sigaction(SIGRT_12, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000053>
14:28:44.475621 rt_sigaction(SIGRT_13, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000056>
14:28:44.475894 rt_sigaction(SIGRT_14, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000079>
14:28:44.476209 rt_sigaction(SIGRT_15, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000055>
14:28:44.476481 rt_sigaction(SIGRT_16, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000055>
14:28:44.476754 rt_sigaction(SIGRT_17, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000054>
14:28:44.477071 rt_sigaction(SIGRT_18, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000054>
14:28:44.477350 rt_sigaction(SIGRT_19, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000054>
14:28:44.477621 rt_sigaction(SIGRT_20, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000055>
14:28:44.477898 rt_sigaction(SIGRT_21, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000046>
14:28:44.478188 rt_sigaction(SIGRT_22, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000078>
14:28:44.478494 rt_sigaction(SIGRT_23, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000078>
14:28:44.478797 rt_sigaction(SIGRT_24, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000074>
14:28:44.479286 rt_sigaction(SIGRT_25, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000076>
14:28:44.479595 rt_sigaction(SIGRT_26, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000078>
14:28:44.479900 rt_sigaction(SIGRT_27, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000082>
14:28:44.480220 rt_sigaction(SIGRT_28, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000078>
14:28:44.480523 rt_sigaction(SIGRT_29, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000078>
14:28:44.480827 rt_sigaction(SIGRT_30, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000114>
14:28:44.481180 rt_sigaction(SIGRT_31, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000076>
14:28:44.481484 rt_sigaction(SIGRT_32, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000078>
14:28:44.481788 rt_sigaction(SIGABRT, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000076>
14:28:44.482118 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000076>
14:28:44.482418 rt_sigaction(SIGIO, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000076>
14:28:44.482720 rt_sigaction(SIGSYS, NULL, {SIG_DFL, [], 0}, 8) = 0 <0.000075>
14:28:44.496140 lseek(4, 0, SEEK_CUR) = 34 <0.000018>
14:28:44.498511 sendto(3, "<12>Dec 5 14:28:44 virt-v2v[233"..., 118,
MSG_NOSIGNAL, NULL, 0) = 118 <0.000135>
14:28:44.499006 sendto(3, "<13>Dec 5 14:28:44 virt-v2v[233"..., 56,
MSG_NOSIGNAL, NULL, 0) = 56 <0.000143>
14:28:44.499587 close(4) = 0 <0.000017>
14:28:44.500350 close(5) = 0 <0.000885>
14:28:44.502847 rt_sigprocmask(SIG_BLOCK, [QUIT], [], 8) = 0 <0.000085>
14:28:44.503111 rt_sigaction(SIGQUIT, {0x3477298660, [], SA_RESTORER, 0x347620f4f0},
{SIG_DFL, [], SA_RESTORER, 0x347620f4f0}, 8) = 0 <0.000015>
14:28:44.503313 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 <0.000015>
14:28:44.503487 rt_sigprocmask(SIG_BLOCK, [INT], [], 8) = 0 <0.000015>
14:28:44.503680 rt_sigaction(SIGINT, {0x3477298660, [], SA_RESTORER, 0x347620f4f0},
{SIG_DFL, [], SA_RESTORER, 0x347620f4f0}, 8) = 0 <0.000015>
14:28:44.503877 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 <0.000020>
14:28:44.504122 getgroups(0, NULL) = 1 <0.000016>
14:28:44.504302 getgroups(1, [36]) = 1 <0.000014>
14:28:44.504483 open("/proc/sys/kernel/ngroups_max", O_RDONLY) = 4
<0.000032>
14:28:44.504680 read(4, "65536\n", 31) = 6 <0.000020>
14:28:44.504842 close(4) = 0 <0.000018>
14:28:44.505049 setgroups(1, [36]) = 0 <0.000018>
14:28:44.505220 setresgid(-1, 36, -1) = 0 <0.000019>
14:28:44.505383 getegid() = 36 <0.000014>
14:28:44.505554 setresuid(-1, 36, -1) = 0 <0.000019>
14:28:44.505717 geteuid() = 36 <0.000014>
14:28:44.505925 pipe([4, 5]) = 0 <0.000036>
14:28:44.506168 clone(child_stack=0,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fe861b939d0) =
26552 <0.001974>
14:28:44.508338 close(5) = 0 <0.000014>
14:28:44.508455 rt_sigaction(SIGINT, {SIG_IGN, [], SA_RESTORER, 0x347620f4f0},
{0x3477298660, [], SA_RESTORER, 0x347620f4f0}, 8) = 0 <0.000014>
14:28:44.508601 rt_sigaction(SIGQUIT, {SIG_IGN, [], SA_RESTORER, 0x347620f4f0},
{0x3477298660, [], SA_RESTORER, 0x347620f4f0}, 8) = 0 <0.000014>
14:28:44.508746 wait4(26552, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) =
26552 <0.569418>
14:28:45.078506 --- {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=26552, si_status=0,
si_utime=0, si_stime=39} (Child exited) ---
14:28:45.078779 rt_sigaction(SIGINT, {0x3477298660, [], SA_RESTORER, 0x347620f4f0}, NULL,
8) = 0 <0.000264>
14:28:45.079238 rt_sigaction(SIGQUIT, {0x3477298660, [], SA_RESTORER, 0x347620f4f0}, NULL,
8) = 0 <0.000094>
14:28:45.079510 read(4, "", 4) = 0 <0.000093>
14:28:45.079744 close(4) = 0 <0.000097>
14:28:45.080136 open("/proc/sys/kernel/ngroups_max", O_RDONLY) = 4
<0.000115>
14:28:45.080399 read(4, "65536\n", 31) = 6 <0.000094>
14:28:45.080633 close(4) = 0 <0.000091>
14:28:45.080886 setgroups(1, [36]) = -1 EPERM (Operation not permitted)
<0.000105>
14:28:45.081263 setresgid(-1, 0, -1) = 0 <0.000095>
14:28:45.081491 getegid() = 0 <0.000088>
14:28:45.081732 setresuid(-1, 0, -1) = 0 <0.000102>
14:28:45.082031 geteuid() = 0 <0.000089>
14:28:45.082278 rt_sigprocmask(SIG_BLOCK, [INT], [], 8) = 0 <0.000096>
14:28:45.082603 rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x347620f4f0},
{0x3477298660, [], SA_RESTORER, 0x347620f4f0}, 8) = 0 <0.000097>
14:28:45.082877 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 <0.000138>
14:28:45.083215 rt_sigprocmask(SIG_BLOCK, [QUIT], [], 8) = 0 <0.000095>
14:28:45.083480 rt_sigaction(SIGQUIT, {SIG_DFL, [], SA_RESTORER, 0x347620f4f0},
{0x3477298660, [], SA_RESTORER, 0x347620f4f0}, 8) = 0 <0.000097>
14:28:45.083754 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 <0.000098>
14:28:45.084272 open("/dev/null", O_RDONLY) = 4 <0.000178>
14:28:45.084574 ioctl(4, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS,
0x7fff884f1a98) = -1 ENOTTY (Inappropriate ioctl for device) <0.000076>
14:28:45.086381 lseek(4, 0, SEEK_CUR) = 0 <0.000028>
14:28:45.086570 fstat(4, {st_mode=S_IFCHR|0666, st_rdev=makedev(1, 3), ...}) = 0
<0.000029>
14:28:45.086774 fcntl(4, F_SETFD, FD_CLOEXEC) = 0 <0.000028>
14:28:45.087449 stat("/tmp/", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=4096, ...})
= 0 <0.000033>
14:28:45.087673 stat("/tmp/", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=4096, ...})
= 0 <0.000031>
14:28:45.088018 open("/tmp/w6utybl9HW", O_RDWR|O_CREAT|O_EXCL|O_NOFOLLOW, 0600)
= 5 <0.000490>
14:28:45.088683 ioctl(5, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS,
0x7fff884f1aa8) = -1 ENOTTY (Inappropriate ioctl for device) <0.000029>
14:28:45.088871 lseek(5, 0, SEEK_CUR) = 0 <0.000029>
14:28:45.089110 fstat(5, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0 <0.000029>
14:28:45.089311 fcntl(5, F_SETFD, FD_CLOEXEC) = 0 <0.000027>
14:28:45.089497 chmod("/tmp/w6utybl9HW", 0600) = 0 <0.000044>
14:28:45.089775 fstat(5, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0 <0.000029>
14:28:45.090044 stat("/tmp/w6utybl9HW", {st_mode=S_IFREG|0600, st_size=0, ...})
= 0 <0.000033>
14:28:45.090275 stat("/tmp/w6utybl9HW", {st_mode=S_IFREG|0600, st_size=0, ...})
= 0 <0.000032>
14:28:45.090567 stat("/tmp/w6utybl9HW", {st_mode=S_IFREG|0600, st_size=0, ...})
= 0 <0.000033>
14:28:45.090786 lstat("/tmp/w6utybl9HW", {st_mode=S_IFREG|0600, st_size=0, ...})
= 0 <0.000032>
14:28:45.091027 unlink("/tmp/w6utybl9HW") = 0 <0.000049>
14:28:45.091234 fstat(5, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0 <0.000028>
14:28:45.091618 close(4) = 0 <0.000032>
14:28:45.091815 pipe([4, 7]) = 0 <0.000066>
14:28:45.092046 ioctl(4, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS,
0x7fff884f1c68) = -1 EINVAL (Invalid argument) <0.000029>
14:28:45.092233 lseek(4, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) <0.000029>
14:28:45.092429 ioctl(7, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS,
0x7fff884f1c68) = -1 EINVAL (Invalid argument) <0.000028>
14:28:45.092612 lseek(7, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) <0.000027>
14:28:45.092791 fcntl(4, F_SETFD, FD_CLOEXEC) = 0 <0.000027>
14:28:45.092997 fcntl(7, F_SETFD, FD_CLOEXEC) = 0 <0.000028>
14:28:45.093198 close(5) = 0 <0.000051>
14:28:45.093404 pipe([5, 8]) = 0 <0.000036>
14:28:45.093598 ioctl(5, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS,
0x7fff884f1c68) = -1 EINVAL (Invalid argument) <0.000029>
14:28:45.093781 lseek(5, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) <0.000027>
14:28:45.094014 ioctl(8, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS,
0x7fff884f1c68) = -1 EINVAL (Invalid argument) <0.000029>
14:28:45.094206 lseek(8, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) <0.000027>
14:28:45.094390 fcntl(5, F_SETFD, FD_CLOEXEC) = 0 <0.000028>
14:28:45.094567 fcntl(8, F_SETFD, FD_CLOEXEC) = 0 <0.000028>
14:28:45.094798 pipe([9, 10]) = 0 <0.000064>
14:28:45.095037 ioctl(9, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS,
0x7fff884f1c68) = -1 EINVAL (Invalid argument) <0.000029>
14:28:45.095225 lseek(9, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) <0.000027>
14:28:45.095413 ioctl(10, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS,
0x7fff884f1c68) = -1 EINVAL (Invalid argument) <0.000028>
14:28:45.095597 lseek(10, 0, SEEK_CUR) = -1 ESPIPE (Illegal seek) <0.000027>
14:28:45.095777 fcntl(9, F_SETFD, FD_CLOEXEC) = 0 <0.000028>
14:28:45.096019 fcntl(10, F_SETFD, FD_CLOEXEC) = 0 <0.000028>
14:28:45.096229 clone(child_stack=0,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fe861b939d0) =
26553 <0.001945>
14:28:45.098477 close(10) = 0 <0.000016>
14:28:45.100108 read(9, "", 8192) = 0 <0.018531>
14:28:45.118890 close(9) = 0 <0.000169>
14:28:45.119270 close(4) = 0 <0.000085>
14:28:45.119528 close(8) = 0 <0.000086>
14:28:45.119847 wait4(26553, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) =
26553 <0.094938>
14:28:45.214994 --- {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=26553, si_status=0,
si_utime=0, si_stime=0} (Child exited) ---
14:28:45.215204 close(7) = 0 <0.000098>
14:28:45.215489 rmdir("/tmp/UBKmK8QbfM") = 0 <0.000312>
14:28:45.216018 close(5) = 0 <0.000098>
14:28:45.232037 exit_group(32) = ?
Process 23339 detached
[root@Fedora16-64P2V ~]#
8:57 p.m.
New subject: P2Vs seem to require a very robust Ethernet
Trying a P2V again, this time using a physical Fedora system for the NFS RHEV Export
domain instead of the Storagetek NFS server. Everything else is the same. Same source
server, same conversion server, same v2v and p2v versions for now. The only variable
different is the RHEV Export NFS server.
Nothing is ever simple. For our new F16 NFS server, with the new systemd, the startup
rules have changed. For anyone else who needs to use an F16 system as an NFS server for
RHEV exports or maybe ISOs, here are some notes. YMMV, but this worked for me. Thanks to
several forums for the raw materials that went into the notes below.
- Greg
****************************************************
First things first. We need a usable text editor.
yum install nano
Now we should change selinux to "permissive".
[root@IT etc]# cd /etc/selinux
[root@IT selinux]# ls
config restorecond.conf restorecond_user.conf semanage.conf targeted
[root@IT selinux]# nano config
[root@IT selinux]# more config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - SELinux is fully disabled.
SELINUX=permissive
# SELINUXTYPE= type of policy in use. Possible values are:
# targeted - Only targeted network daemons are protected.
# strict - Full SELinux protection.
SELINUXTYPE=targeted
[root@IT selinux]#
Now install the packages we need for NFS.
yum install nfs-utils (This should pull in all the dependencies including rpcbind.)
yum install system-config-nfs
yum install system-config-nfs-docs
Set them to start at boot time. And open up all the firewall rules.
systemctl enable rpcbind.service
systemctl enable nfs-server.service
systemctl disable iptables.service
Start everything up for now.
systemctl stop iptables.service
systemsctl start rpcbind.service
systemctl start nfs-server.service
And now we can set up our NFS share and edit /etc/exports. Where is the best place to put
the NFS share we want?
[root@IT /]# df -h
Filesystem Size Used Avail Use% Mounted on
rootfs 50G 2.7G 47G 6% /
devtmpfs 1.5G 0 1.5G 0% /dev
tmpfs 1.5G 240K 1.5G 1% /dev/shm
tmpfs 1.5G 40M 1.4G 3% /run
/dev/mapper/vg_it-lv_root 50G 2.7G 47G 6% /
tmpfs 1.5G 40M 1.4G 3% /run
tmpfs 1.5G 0 1.5G 0% /sys/fs/cgroup
tmpfs 1.5G 0 1.5G 0% /media
/dev/sde3 485M 56M 404M 13% /boot
/dev/mapper/vg_it-lv_home 212G 252M 201G 1% /home
[root@IT /]# ls /home
fredy lost+found
[root@IT /]# mkdir /home/RHEVexports
[root@IT /]# chown 36.36 /home/RHEVexports
[root@IT /]# ls -al /home total 32
drwxr-xr-x. 5 root root 4096 Dec 5 20:12 .
dr-xr-xr-x. 21 root root 4096 Dec 5 11:49 ..
drwx------. 25 fredy fredy 4096 Dec 5 11:49 fredy
drwx------. 2 root root 16384 Nov 29 06:32 lost+found
drwxr-xr-x. 2 36 36 4096 Dec 5 20:12 RHEVexports
[root@IT etc]# nano /etc/exports
[root@IT etc]# more /etc/exports
/home/RHEVexports *(rw,sync,nohide,no_root_squash)
[root@IT etc]# exportfs -av
exporting *:/home/RHEVexports
[root@IT etc]#
We decided /home/RHEVexports seems like a reasonable location. And now it's mountable
from a different system:
[root@Fedora16-64P2V log]# mkdir /mnt/stuff
[root@Fedora16-64P2V log]# mount -t nfs 175.10.0.95:/home/RHEVexports /mnt/stuff
[root@Fedora16-64P2V log]# cd /mnt/stuff
[root@Fedora16-64P2V stuff]# touch a.a
[root@Fedora16-64P2V stuff]# rm a.a
rm: remove regular empty file `a.a'? y
[root@Fedora16-64P2V stuff]# cd /
[root@Fedora16-64P2V /]# umount /mnt/stuff [root@Fedora16-64P2V /]#
Should be ready to use as the RHEV-M export domain now.
4616
days inactive
4620
days old
17 comments
2 participants
participants (2)
-
Fredy Hernández -
Greg Scott