Discussion:
KVM + LXC on the same host
ilya musayev
2014-06-05 00:58:15 UTC
Permalink
We are considering running KVM and LXC on the same host and hopefully
control both through cloudstack.

I know there are agents involved for each component, i dont know if we
can have a hybrid of LXC+KVM.

The use case is simple, we would like the end user to pick LXC/Docker
for performance, or KVM instance if he really needed all bells and
whistles of dedicated kernel in fully virtualized environment.

Is anyone aware why we should not mix 2 workloads on the same host? Is
it possible at this point in time to mix LXC, KVM and CloudStack, i
assume the answer is no, but perhaps there is a hack i can try.

Thanks
ilya
Nguyen Anh Tu
2014-06-06 14:53:11 UTC
Permalink
That should be a good idea, Ilya. At that moment, i'm working on Docker
support. When it's done, we can run only Docker hosts, no need to use KVM
for hosting system vms.

Cheers,
--Tuna

Sent from my GT-N7000
Post by ilya musayev
We are considering running KVM and LXC on the same host and hopefully
control both through cloudstack.
I know there are agents involved for each component, i dont know if we can
have a hybrid of LXC+KVM.
The use case is simple, we would like the end user to pick LXC/Docker for
performance, or KVM instance if he really needed all bells and whistles of
dedicated kernel in fully virtualized environment.
Is anyone aware why we should not mix 2 workloads on the same host? Is it
possible at this point in time to mix LXC, KVM and CloudStack, i assume the
answer is no, but perhaps there is a hack i can try.
Thanks
ilya
Pierre-Luc Dion
2014-06-06 15:42:01 UTC
Permalink
ilya,

Let us know how it goes your hybrid of LXC+KVM. I'm interested to know how
it's going, I might try that too on the side.




Pierre-Luc Dion
Architecte de Solution Cloud | Cloud Solutions Architect
855-OK-CLOUD (855-652-5683) x1101
- - -

*CloudOps*420 rue Guy
Montréal QC H3J 1S6
www.cloudops.com
@CloudOps_
Post by Nguyen Anh Tu
That should be a good idea, Ilya. At that moment, i'm working on Docker
support. When it's done, we can run only Docker hosts, no need to use KVM
for hosting system vms.
Cheers,
--Tuna
Sent from my GT-N7000
Post by ilya musayev
We are considering running KVM and LXC on the same host and hopefully
control both through cloudstack.
I know there are agents involved for each component, i dont know if we
can
Post by ilya musayev
have a hybrid of LXC+KVM.
The use case is simple, we would like the end user to pick LXC/Docker for
performance, or KVM instance if he really needed all bells and whistles
of
Post by ilya musayev
dedicated kernel in fully virtualized environment.
Is anyone aware why we should not mix 2 workloads on the same host? Is it
possible at this point in time to mix LXC, KVM and CloudStack, i assume
the
Post by ilya musayev
answer is no, but perhaps there is a hack i can try.
Thanks
ilya
ilya musayev
2014-06-06 20:20:13 UTC
Permalink
Tuna

Thanks for the feedback, conceptually, i was not trying to address the
issue of sysvms not running on LXC.

Here is the use case as i see it.

Assume you roll out a farm of 10 KVM servers and your density with KVM
is 200 virtual machines. Most of these VMs dont require independent
kernel and additional layers of virtualization abstraction. As the
result, you can place 400 LXC machines and 20 fully virtualized Linux or
even Windows Servers. If you do chargeback, you can offer the LXC
machines for much lower price point since we can place more LXC containers.

Your density becomes much greater with LXC and yet you still cover the
corner case when end-user needs KVM instance.

Pierre-Luc

If I will get to try this scenario - i may have to alter the KVM/LXC
agent somewhat to make it work with CloudStack, i will let you know.
Alternative solution would be to run separate LXC and KVM clusters,
which is also a possibility - but usage and distribution will be uneven.

Regards
ilya

The concept of combining both technologies under one hypervisor
Post by Pierre-Luc Dion
ilya,
Let us know how it goes your hybrid of LXC+KVM. I'm interested to know how
it's going, I might try that too on the side.
Pierre-Luc Dion
Architecte de Solution Cloud | Cloud Solutions Architect
855-OK-CLOUD (855-652-5683) x1101
- - -
*CloudOps*420 rue Guy
Montréal QC H3J 1S6
www.cloudops.com
@CloudOps_
Post by Nguyen Anh Tu
That should be a good idea, Ilya. At that moment, i'm working on Docker
support. When it's done, we can run only Docker hosts, no need to use KVM
for hosting system vms.
Cheers,
--Tuna
Sent from my GT-N7000
Post by ilya musayev
We are considering running KVM and LXC on the same host and hopefully
control both through cloudstack.
I know there are agents involved for each component, i dont know if we
can
Post by ilya musayev
have a hybrid of LXC+KVM.
The use case is simple, we would like the end user to pick LXC/Docker for
performance, or KVM instance if he really needed all bells and whistles
of
Post by ilya musayev
dedicated kernel in fully virtualized environment.
Is anyone aware why we should not mix 2 workloads on the same host? Is it
possible at this point in time to mix LXC, KVM and CloudStack, i assume
the
Post by ilya musayev
answer is no, but perhaps there is a hack i can try.
Thanks
ilya
Kishan Kavala
2014-06-09 06:59:52 UTC
Permalink
-----Original Message-----
Sent: Saturday, 7 June 2014 1:50 AM
Subject: Re: KVM + LXC on the same host
Tuna
Thanks for the feedback, conceptually, i was not trying to address the issue of
sysvms not running on LXC.
Here is the use case as i see it.
Assume you roll out a farm of 10 KVM servers and your density with KVM is
200 virtual machines. Most of these VMs dont require independent kernel
and additional layers of virtualization abstraction. As the result, you can place
400 LXC machines and 20 fully virtualized Linux or even Windows Servers. If
you do chargeback, you can offer the LXC machines for much lower price point
since we can place more LXC containers.
Your density becomes much greater with LXC and yet you still cover the
corner case when end-user needs KVM instance.
Pierre-Luc
If I will get to try this scenario - i may have to alter the KVM/LXC agent
somewhat to make it work with CloudStack, i will let you know.
Ilya,
Some minor changes to KVM/LXC agent will be required to make this work. Below piece of code in createVMFromSpec (LibvirtComputingResource), deploys systems Vms in KVM and user Vms in LXC.
If we include some flag in VirtualMachineTO, it should be possible to specify which userVm has to be deployed (KVM/LXC).

if (HypervisorType.LXC == _hypervisorType && VirtualMachine.Type.User == vmTO.getType()) {
// LXC domain is only valid for user VMs. Use KVM for system VMs.
guest.setGuestType(GuestDef.guestType.LXC);
vm.setHvsType(HypervisorType.LXC.toString().toLowerCase());
} else {
guest.setGuestType(GuestDef.guestType.KVM);
vm.setHvsType(HypervisorType.KVM.toString().toLowerCase());
vm.setLibvirtVersion(_hypervisorLibvirtVersion);
vm.setQemuVersion(_hypervisorQemuVersion);
}
Alternative solution would be to run separate LXC and KVM clusters, which is
also a possibility - but usage and distribution will be uneven.
Regards
ilya
The concept of combining both technologies under one hypervisor On 6/6/14,
Post by Pierre-Luc Dion
ilya,
Let us know how it goes your hybrid of LXC+KVM. I'm interested to know
how it's going, I might try that too on the side.
Pierre-Luc Dion
Architecte de Solution Cloud | Cloud Solutions Architect 855-OK-CLOUD
(855-652-5683) x1101
- - -
*CloudOps*420 rue Guy
Montréal QC H3J 1S6
www.cloudops.com
@CloudOps_
Post by Nguyen Anh Tu
That should be a good idea, Ilya. At that moment, i'm working on
Docker support. When it's done, we can run only Docker hosts, no need
to use KVM for hosting system vms.
Cheers,
--Tuna
Sent from my GT-N7000
Post by ilya musayev
We are considering running KVM and LXC on the same host and
hopefully control both through cloudstack.
I know there are agents involved for each component, i dont know if we
can
Post by ilya musayev
have a hybrid of LXC+KVM.
The use case is simple, we would like the end user to pick
LXC/Docker for performance, or KVM instance if he really needed all
bells and whistles
of
Post by ilya musayev
dedicated kernel in fully virtualized environment.
Is anyone aware why we should not mix 2 workloads on the same host?
Is it possible at this point in time to mix LXC, KVM and CloudStack,
i assume
the
Post by ilya musayev
answer is no, but perhaps there is a hack i can try.
Thanks
ilya
Chiradeep Vittal
2014-06-11 00:26:30 UTC
Permalink
I thought the LXC 2.0 project uses KVM system vms on the same host as LXC containers?
https://cwiki.apache.org/confluence/x/oJNMAg

From: Kishan Kavala <Kishan.Kavala-Sxgqhf6Nn4DQT0dZR+***@public.gmane.org<mailto:***@citrix.com>>
Reply-To: "dev-***@public.gmane.org<mailto:dev-***@public.gmane.org>" <***@cloudstack.apache.org<mailto:dev-***@public.gmane.org>>
Date: Sunday, June 8, 2014 at 11:59 PM
To: "dev-***@public.gmane.org<mailto:dev-***@public.gmane.org>" <***@cloudstack.apache.org<mailto:dev-***@public.gmane.org>>
Subject: RE: KVM + LXC on the same host



-----Original Message-----
From: ilya musayev [mailto:ilya.mailing.lists-***@public.gmane.org]
Sent: Saturday, 7 June 2014 1:50 AM
To: dev-***@public.gmane.org<mailto:dev-***@public.gmane.org>
Subject: Re: KVM + LXC on the same host
Tuna
Thanks for the feedback, conceptually, i was not trying to address the issue of
sysvms not running on LXC.
Here is the use case as i see it.
Assume you roll out a farm of 10 KVM servers and your density with KVM is
200 virtual machines. Most of these VMs dont require independent kernel
and additional layers of virtualization abstraction. As the result, you can place
400 LXC machines and 20 fully virtualized Linux or even Windows Servers. If
you do chargeback, you can offer the LXC machines for much lower price point
since we can place more LXC containers.
Your density becomes much greater with LXC and yet you still cover the
corner case when end-user needs KVM instance.
Pierre-Luc
If I will get to try this scenario - i may have to alter the KVM/LXC agent
somewhat to make it work with CloudStack, i will let you know.
Ilya,
Some minor changes to KVM/LXC agent will be required to make this work. Below piece of code in createVMFromSpec (LibvirtComputingResource), deploys systems Vms in KVM and user Vms in LXC.
If we include some flag in VirtualMachineTO, it should be possible to specify which userVm has to be deployed (KVM/LXC).

if (HypervisorType.LXC == _hypervisorType && VirtualMachine.Type.User == vmTO.getType()) {
// LXC domain is only valid for user VMs. Use KVM for system VMs.
guest.setGuestType(GuestDef.guestType.LXC);
vm.setHvsType(HypervisorType.LXC.toString().toLowerCase());
} else {
guest.setGuestType(GuestDef.guestType.KVM);
vm.setHvsType(HypervisorType.KVM.toString().toLowerCase());
vm.setLibvirtVersion(_hypervisorLibvirtVersion);
vm.setQemuVersion(_hypervisorQemuVersion);
}



Alternative solution would be to run separate LXC and KVM clusters, which is
also a possibility - but usage and distribution will be uneven.
Regards
ilya
The concept of combining both technologies under one hypervisor On 6/6/14,
Post by Pierre-Luc Dion
ilya,
Let us know how it goes your hybrid of LXC+KVM. I'm interested to know
how it's going, I might try that too on the side.
Pierre-Luc Dion
Architecte de Solution Cloud | Cloud Solutions Architect 855-OK-CLOUD
(855-652-5683) x1101
- - -
*CloudOps*420 rue Guy
Montréal QC H3J 1S6
www.cloudops.com
@CloudOps_
Post by Nguyen Anh Tu
That should be a good idea, Ilya. At that moment, i'm working on
Docker support. When it's done, we can run only Docker hosts, no need
to use KVM for hosting system vms.
Cheers,
--Tuna
Sent from my GT-N7000
Post by ilya musayev
We are considering running KVM and LXC on the same host and
hopefully control both through cloudstack.
I know there are agents involved for each component, i dont know if we
can
Post by ilya musayev
have a hybrid of LXC+KVM.
The use case is simple, we would like the end user to pick
LXC/Docker for performance, or KVM instance if he really needed all
bells and whistles
of
Post by ilya musayev
dedicated kernel in fully virtualized environment.
Is anyone aware why we should not mix 2 workloads on the same host?
Is it possible at this point in time to mix LXC, KVM and CloudStack,
i assume
the
Post by ilya musayev
answer is no, but perhaps there is a hack i can try.
Thanks
ilya
Kishan Kavala
2014-06-11 11:44:32 UTC
Permalink
KVM system vms on the same host as LXC containers is already supported.
Below proposal was to support userVms in a similar way.
-----Original Message-----
Sent: Wednesday, 11 June 2014 5:57 AM
Subject: Re: KVM + LXC on the same host
I thought the LXC 2.0 project uses KVM system vms on the same host as LXC containers?
https://cwiki.apache.org/confluence/x/oJNMAg
From: Kishan Kavala
Date: Sunday, June 8, 2014 at 11:59 PM
Subject: RE: KVM + LXC on the same host
-----Original Message-----
Sent: Saturday, 7 June 2014 1:50 AM
Subject: Re: KVM + LXC on the same host
Tuna
Thanks for the feedback, conceptually, i was not trying to address the issue of
sysvms not running on LXC.
Here is the use case as i see it.
Assume you roll out a farm of 10 KVM servers and your density with KVM is
200 virtual machines. Most of these VMs dont require independent kernel
and additional layers of virtualization abstraction. As the result, you can place
400 LXC machines and 20 fully virtualized Linux or even Windows Servers. If
you do chargeback, you can offer the LXC machines for much lower price point
since we can place more LXC containers.
Your density becomes much greater with LXC and yet you still cover the
corner case when end-user needs KVM instance.
Pierre-Luc
If I will get to try this scenario - i may have to alter the KVM/LXC agent
somewhat to make it work with CloudStack, i will let you know.
Ilya,
Some minor changes to KVM/LXC agent will be required to make this work.
Below piece of code in createVMFromSpec (LibvirtComputingResource),
deploys systems Vms in KVM and user Vms in LXC.
If we include some flag in VirtualMachineTO, it should be possible to specify
which userVm has to be deployed (KVM/LXC).
if (HypervisorType.LXC == _hypervisorType && VirtualMachine.Type.User
== vmTO.getType()) {
// LXC domain is only valid for user VMs. Use KVM for system VMs.
guest.setGuestType(GuestDef.guestType.LXC);
vm.setHvsType(HypervisorType.LXC.toString().toLowerCase());
} else {
guest.setGuestType(GuestDef.guestType.KVM);
vm.setHvsType(HypervisorType.KVM.toString().toLowerCase());
vm.setLibvirtVersion(_hypervisorLibvirtVersion);
vm.setQemuVersion(_hypervisorQemuVersion);
}
Alternative solution would be to run separate LXC and KVM clusters, which is
also a possibility - but usage and distribution will be uneven.
Regards
ilya
The concept of combining both technologies under one hypervisor On 6/6/14,
Post by Pierre-Luc Dion
ilya,
Let us know how it goes your hybrid of LXC+KVM. I'm interested to know
how it's going, I might try that too on the side.
Pierre-Luc Dion
Architecte de Solution Cloud | Cloud Solutions Architect 855-OK-CLOUD
(855-652-5683) x1101
- - -
*CloudOps*420 rue Guy
Montréal QC H3J 1S6
www.cloudops.com
@CloudOps_
On Fri, Jun 6, 2014 at 10:53 AM, Nguyen Anh Tu
Post by Nguyen Anh Tu
That should be a good idea, Ilya. At that moment, i'm working on
Docker support. When it's done, we can run only Docker hosts, no need
to use KVM for hosting system vms.
Cheers,
--Tuna
Sent from my GT-N7000
On Jun 5, 2014 7:58 AM, "ilya musayev"
Post by ilya musayev
We are considering running KVM and LXC on the same host and
hopefully control both through cloudstack.
I know there are agents involved for each component, i dont know if we
can
Post by ilya musayev
have a hybrid of LXC+KVM.
The use case is simple, we would like the end user to pick
LXC/Docker for performance, or KVM instance if he really needed all
bells and whistles
of
Post by ilya musayev
dedicated kernel in fully virtualized environment.
Is anyone aware why we should not mix 2 workloads on the same host?
Is it possible at this point in time to mix LXC, KVM and CloudStack,
i assume
the
Post by ilya musayev
answer is no, but perhaps there is a hack i can try.
Thanks
ilya
John Kinsella
2014-06-07 02:38:53 UTC
Permalink
Hey Ilya -

So, for about a month now we’ve had a system running SmartOS[1], which gives a combination of containers and KVM, albeit on illumos instead of linux. In general I’m not impressed by SmartOS’s story, but we had one customer asking for it (and I sorta expect more, we’re not officially supporting it yet). It’s a little early for me to even ponder integrating it with ACS, but might but worth a glance if you’re looking for that KVM/container mix.

Downsides:
* Compared to a Linux hypervisor, much less feature support (missing Ceph is personally bugging me)
* It’s…Solaris (the OS that won’t go away, c0t0d0s0 gives me flashbacks from 10 years ago). For us our infrastructure/automation is fairly tuned for linux, so really embracing this will require…effort.

John
1: http://smartos.org/
We are considering running KVM and LXC on the same host and hopefully control both through cloudstack.
I know there are agents involved for each component, i dont know if we can have a hybrid of LXC+KVM.
The use case is simple, we would like the end user to pick LXC/Docker for performance, or KVM instance if he really needed all bells and whistles of dedicated kernel in fully virtualized environment.
Is anyone aware why we should not mix 2 workloads on the same host? Is it possible at this point in time to mix LXC, KVM and CloudStack, i assume the answer is no, but perhaps there is a hack i can try.
Thanks
ilya
Nux!
2014-06-09 11:12:03 UTC
Permalink
Post by John Kinsella
Hey Ilya -
So, for about a month now we’ve had a system running SmartOS[1],
which gives a combination of containers and KVM, albeit on illumos
instead of linux. In general I’m not impressed by SmartOS’s story, but
we had one customer asking for it (and I sorta expect more, we’re not
officially supporting it yet). It’s a little early for me to even
ponder integrating it with ACS, but might but worth a glance if you’re
looking for that KVM/container mix.
* Compared to a Linux hypervisor, much less feature support (missing
Ceph is personally bugging me)
* It’s…Solaris (the OS that won’t go away, c0t0d0s0 gives me
flashbacks from 10 years ago). For us our infrastructure/automation is
fairly tuned for linux, so really embracing this will require…effort.
Check out the Proxmox project which provides both containers and VMs,
in the form of OpenVZ (not ideal, I know) and KVM.

Lucian
--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro
Loading...