controlling cpu usage part 8: Adding Pools to a Zone

Zones are able to use the pool subsystem directly. When a zone is defined it can be associated with a named pool by setting the zone's pool property to the name of an existing pool.

 # zonecfg -z zone set pool=pool_web

Multiple zones may share the same pool. In this case each zone should set the cpu-shares resource type to arbitrate between the relative use of CPU for each zone in the pool.

 # zonecfg -z test0z1 set cpu-shares=20
 # zonecfg -z test0z2 set cpu-shares=30

Solaris 11/06 introduces the concept of anoymous pools. These are pools created by a zone when it boots for the exclusive use of that zone. This is done through the dedicated-cpu resource type for a zone. The dedicated-cpu resource type has two properties, ncpus which indicates the number CPU's to put into the created pool, or a range of CPU's if a dynamic pool is desired, and importance which sets the pool.importance property in the pool for use as tie-breaker by poold.

 # zonecfg -z test0z1
 zonecfg:test0z1> add dedicated-cpu
 zonecfg:test0z1:dedicated-cpu> set ncpu=1-3
 zonecfg:test0z1:dedicated-cpu> set importance=10
 zonecfg:test0z1:dedicated-cpu> end
 zonecfg:test0z1> commit
 zonecfg:test0z1> exit

Whenever the zone boots the zoneadmd deamon will create a pool and assign the zone to the pool. 

Note that the dedicated-cpu resource on a zone means that the pool cannot be shared between multiple zones.

If the FSS is desired for the zone then the scheduling-class resource type for the zone should also be set.

 # zonecfg -z test0z1
 zonecfg:test0z1> set scheduling-class=FSS
 zonecfg:test0z1> commit
 zonecfg:test0z1> exit

Note The pools system must be already configured on the system before the dedicated-cpu resource type is used by the zone. If the pool system is not configured any attempt to boot the pool will result in an error from zoneadm.

If there is not enough resources to create the pool an attempt to boot results in a fatal error, and the boot fails.

 # zoneadm -z test0z1 boot
 zoneadm: zone 'test0z1': libpool(3LIB) error: invalid configuration
 zoneadm: zone 'test0z1': dedicated-cpu setting cannot be instatiated
 zoneadm: zone 'test0z1': call to zoneadmd failed

When the zone is booted a temporary pool called SUNWtmp_zonename is created.

        pool SUNWtmp_test0z1
                int     pool.sys_id 4
                boolean pool.active true
                boolean pool.default false
                int     pool.importance 10
                string  pool.comment
                boolean pool.temporary true
                pset    SUNWtmp_test0z1

        pset SUNWtmp_test0z1
                int     pset.sys_id 1
                boolean pset.defaul false
                uint    pset.min 1
                uint    pset.max 3
                string  pset.units population
                uint    pset.load 1991
                uint    pset.size 1
                string  pset.comment
                boolean pset.temporary true

                cpu
                        int     cpu.sys_id 0
                        string  cpu.comment
                        string  cpu.status on-line

The dedicated-cpu resource type creates a pool for the exclusive use of this zone. The zone has exclusive access to the CPU's in the pool. For that reason the cpu-shares resource type in the zone has no meaning if a dedicated-cpu resource type is also defined. The zone will always have 100% of the shares in the processor set, and so will always have the entire processor set to itself irrespective of the number of shares. 

controlling cpu usage part 7: Pools

Dynamic pools were introduced in Solaris 10. A pool is binding of a resource and processor set together into a persistent entry. A pool allows us to name a pool and assign resource controls, such as the scheduler, on a persistent basis. 

Pools can also be used for projects using the project.pool attribute in /etc/project
By default if the pools system is enabled using SMF there is a default processor set created whcih is attached to a default pool. This configuration can be viewed using the poolcfg -dc info command.

 # poolcfg -dc info
 poolcfg: cannot load configuration from /dev/poolctl: Facility is not active

 # svcadm enable svcs:/system/pools:default
 # poolcfg -dc info

 system default
         string  system.comment
         int     system.version 1
         boolean system.bind-default true
         string  system.poold.objectives wt-load

         pool pool_default
                 int     pool.sys_id 0
                 boolean pool.active true
                 boolean pool.default true
                 int     pool.importance 1
                 string  pool.comment
                 pset    pset_default

         pset pset_default
                 int     pset.sys_id -1
                 boolean pset.default true
                 uint    pset.min 1
                 uint    pset.max 65536
                 string  pset.units population
                 uint    pset.load 481
                 uint    pset.size 2
                 string  pset.comment

                 cpu
                         int     cpu.sys_id 1
                         string  cpu.comment
                         string  cpu.status on-line

                 cpu
                         int     cpu.sys_id 0
                         string  cpu.comment
                         string  cpu.status on-line

To configure pools on a system you must create a configuration file. By default this file should be named /etc/pooladm.conf so it would automatically load at boot time. The easiest way of creating a file is to configure the current system as desired and then perform a pooladm save command.

 # pooladm -s /etc/pooladm.conf

The following example saves the current kernel state as /etc/pooladm.conf, and then uses poolcfg to create a new pool called pool_web which contains one processor set pset_web which has one CPU.

 # pooladm -s /etc/pooladm.conf
 # poolcfg -c 'create pool pool_web'
 # poolcfg -c 'create pset pset_web (uint pset.min = 1; uint pset.max = 4)'
 # poolcfg -c 'associate pool pool_web (pset pset_web)'
 # pooladm -c
 # pooladm -s

We can then display the resultant condiguration.

 # poolcfg -dc info

 system default
         string  system.comment
         int     system.version 1
         boolean system.bind-default true
         string  system.poold.objectives wt-load

         pool pool_web
                 int     pool.sys_id 1
                 boolean pool.active true
                 boolean pool.default false
                 int     pool.importance 1
                 string  pool.comment
                 pset    pset_web
       

         pool pool_default
                 int     pool.sys_id 0
                 boolean pool.active true
                 boolean pool.default true
                 int     pool.importance 1
                 string  pool.comment
                 pset    pset_default

         pset pset_web
                 int     pset.sys_id 1
                 boolean pset.default false
                 uint    pset.min 1
                 uint    pset.max 4
                 string  pset.units population
                 uint    pset.load 0
                 uint    pset.size 1
                 string  pset.comment

                 cpu
                         int     cpu.sys_id 0
                         string  cpu.comment
                         string  cpu.status on-line
       

         pset pset_default
                 int     pset.sys_id -1
                 boolean pset.default true
                 uint    pset.min 1
                 uint    pset.max 65536
                 string  pset.units population
                 uint    pset.load 0
                 uint    pset.size 1
                 string  pset.comment


                 cpu
                         int     cpu.sys_id 1
                         string  cpu.comment
                         string  cpu.status on-line

Note that the default pool and the default pset have their default property set to true.

We can also define other resource properties for the pools. To do this we can also define and set the pool.scheduler to the pool.

The following example sets the FSS scheduler for the pool pool_web

 # poolcfg -c 'modify pool pool_web (string pool.scheduler="FSS")'
 # poolcfg -dc info

 system default
         string  system.comment
         int     system.version 1
         boolean system.bind-default true
         string  system.poold.objectives wt-load

         pool pool_web
                 int     pool.sys_id 1
                 boolean pool.active true
                 boolean pool.default false
                 string  pool.scheduler FSS
                 int     pool.importance 1
                 string  pool.comment
                 pset    pset_web

 ...

As the load in one processor set increases the number of CPU's in that pool is increased by taking CPU's from other pools. The pset.min and pset.max properties of the processor set are used to constrain the minimum and maximum number of CPU's that can exist in a pool.

If the there is a tie for resource the pool.importance property is used as a tie-breaker.

To enable dynamic pools the svc:/system/pools/dynamic:default service must be enabled. This will start the poold deamon which performs the dynamic modification of the processor sets on the system.

 # ps -eaf|grep poold
     root 20334  3948   0 12:23:51 pts/4       0:00 grep poold

 # svcadm enable svc:/system/pools/dynamic:default
 # ps -eaf|grep poold
     root 20423  3948   0 12:24:55 pts/4       0:00 grep poold
     root 20422     1   0 12:24:53 ?           0:00 /usr/lib/pool/poold