<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>Cgroups on avni.sh</title>
    <link>http://www.avni.sh/tags/cgroups/</link>
    <description>Recent content in Cgroups on avni.sh</description>
    <image>
      <title>avni.sh</title>
      <url>http://www.avni.sh/cover.webp</url>
      <link>http://www.avni.sh/cover.webp</link>
    </image>
    <generator>Hugo</generator>
    <language>en</language>
    <lastBuildDate>Fri, 27 Jan 2023 00:00:00 +0000</lastBuildDate>
    <atom:link href="http://www.avni.sh/tags/cgroups/index.xml" rel="self" type="application/rss+xml" />
    <item>
      <title>Container Architecture</title>
      <link>http://www.avni.sh/posts/cloud-native/container-architecture/</link>
      <pubDate>Fri, 27 Jan 2023 00:00:00 +0000</pubDate>
      <guid>http://www.avni.sh/posts/cloud-native/container-architecture/</guid>
      <description>Creation and execution of container is enabled using features like namespaces, Cgroups, seccomp, and SELinux</description>
      <content:encoded><![CDATA[<p>To isolate the processes running inside a <a href="/posts/computer-science/technologies/cloud-native/containers/" target="_blank">container</a> from its host system, container engine uses the following four features:</p>
<ul>
<li>Namespaces</li>
<li>Control Groups</li>
<li>Secure Computing</li>
<li>Security-Enhanced Linux</li>
</ul>
<h2 id="namespaces">Namespaces</h2>
<p><strong>Namespaces</strong> are created to limit the reach of a container to its host&rsquo;s resources. It helps with security and well as limits resources available to the container.</p>
<p>Linux command <code>lsns</code> could be used for listing details of namespaces.</p>
<p>The namespaces essential for containers are User, Mount, Unix Timesharing System, Process ID, Network, and Inter-Process Communication.</p>
<h3 id="user">User</h3>
<p>The users and groups created inside a container are different from its host. Processes running inside the container as a <code>root</code> user could be mapped to a non-root user on the host.</p>
<p>Using the <code>id</code> command you can verify that the containers are present on a different user namespace than other processes on your host.</p>
<p>Running <code>id</code> on host:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">$ id
</span></span><span class="line"><span class="cl"><span class="nv">uid</span><span class="o">=</span>1000<span class="o">(</span>username<span class="o">)</span> <span class="nv">gid</span><span class="o">=</span>1000<span class="o">(</span>username<span class="o">)</span> <span class="nv">groups</span><span class="o">=</span>1000<span class="o">(</span>username<span class="o">)</span>,10<span class="o">(</span>wheel<span class="o">)</span> <span class="nv">context</span><span class="o">=</span>unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
</span></span></code></pre></div><p>Running <code>id</code> inside a container created from <code>docker.io/library/httpd:latest</code> container image:</p>



<div class="goat svg-container ">
  
    <svg
      xmlns="http://www.w3.org/2000/svg"
      font-family="Menlo,Lucida Console,monospace"
      
        viewBox="0 0 328 41"
      >
      <g transform='translate(8,16)'>
<text text-anchor='middle' x='0' y='4' fill='currentColor' style='font-size:1em'>r</text>
<text text-anchor='middle' x='0' y='20' fill='currentColor' style='font-size:1em'>u</text>
<text text-anchor='middle' x='8' y='4' fill='currentColor' style='font-size:1em'>o</text>
<text text-anchor='middle' x='8' y='20' fill='currentColor' style='font-size:1em'>i</text>
<text text-anchor='middle' x='16' y='4' fill='currentColor' style='font-size:1em'>o</text>
<text text-anchor='middle' x='16' y='20' fill='currentColor' style='font-size:1em'>d</text>
<text text-anchor='middle' x='24' y='4' fill='currentColor' style='font-size:1em'>t</text>
<text text-anchor='middle' x='24' y='20' fill='currentColor' style='font-size:1em'>=</text>
<text text-anchor='middle' x='32' y='4' fill='currentColor' style='font-size:1em'>@</text>
<text text-anchor='middle' x='32' y='20' fill='currentColor' style='font-size:1em'>0</text>
<text text-anchor='middle' x='40' y='4' fill='currentColor' style='font-size:1em'>1</text>
<text text-anchor='middle' x='40' y='20' fill='currentColor' style='font-size:1em'>(</text>
<text text-anchor='middle' x='48' y='4' fill='currentColor' style='font-size:1em'>4</text>
<text text-anchor='middle' x='48' y='20' fill='currentColor' style='font-size:1em'>r</text>
<text text-anchor='middle' x='56' y='4' fill='currentColor' style='font-size:1em'>e</text>
<text text-anchor='middle' x='56' y='20' fill='currentColor' style='font-size:1em'>o</text>
<text text-anchor='middle' x='64' y='4' fill='currentColor' style='font-size:1em'>d</text>
<text text-anchor='middle' x='64' y='20' fill='currentColor' style='font-size:1em'>o</text>
<text text-anchor='middle' x='72' y='4' fill='currentColor' style='font-size:1em'>7</text>
<text text-anchor='middle' x='72' y='20' fill='currentColor' style='font-size:1em'>t</text>
<text text-anchor='middle' x='80' y='4' fill='currentColor' style='font-size:1em'>2</text>
<text text-anchor='middle' x='80' y='20' fill='currentColor' style='font-size:1em'>)</text>
<text text-anchor='middle' x='88' y='4' fill='currentColor' style='font-size:1em'>a</text>
<text text-anchor='middle' x='96' y='4' fill='currentColor' style='font-size:1em'>f</text>
<text text-anchor='middle' x='96' y='20' fill='currentColor' style='font-size:1em'>g</text>
<text text-anchor='middle' x='104' y='4' fill='currentColor' style='font-size:1em'>d</text>
<text text-anchor='middle' x='104' y='20' fill='currentColor' style='font-size:1em'>i</text>
<text text-anchor='middle' x='112' y='4' fill='currentColor' style='font-size:1em'>6</text>
<text text-anchor='middle' x='112' y='20' fill='currentColor' style='font-size:1em'>d</text>
<text text-anchor='middle' x='120' y='4' fill='currentColor' style='font-size:1em'>2</text>
<text text-anchor='middle' x='120' y='20' fill='currentColor' style='font-size:1em'>=</text>
<text text-anchor='middle' x='128' y='4' fill='currentColor' style='font-size:1em'>e</text>
<text text-anchor='middle' x='128' y='20' fill='currentColor' style='font-size:1em'>0</text>
<text text-anchor='middle' x='136' y='4' fill='currentColor' style='font-size:1em'>:</text>
<text text-anchor='middle' x='136' y='20' fill='currentColor' style='font-size:1em'>(</text>
<text text-anchor='middle' x='144' y='4' fill='currentColor' style='font-size:1em'>/</text>
<text text-anchor='middle' x='144' y='20' fill='currentColor' style='font-size:1em'>r</text>
<text text-anchor='middle' x='152' y='4' fill='currentColor' style='font-size:1em'>u</text>
<text text-anchor='middle' x='152' y='20' fill='currentColor' style='font-size:1em'>o</text>
<text text-anchor='middle' x='160' y='4' fill='currentColor' style='font-size:1em'>s</text>
<text text-anchor='middle' x='160' y='20' fill='currentColor' style='font-size:1em'>o</text>
<text text-anchor='middle' x='168' y='4' fill='currentColor' style='font-size:1em'>r</text>
<text text-anchor='middle' x='168' y='20' fill='currentColor' style='font-size:1em'>t</text>
<text text-anchor='middle' x='176' y='4' fill='currentColor' style='font-size:1em'>/</text>
<text text-anchor='middle' x='176' y='20' fill='currentColor' style='font-size:1em'>)</text>
<text text-anchor='middle' x='184' y='4' fill='currentColor' style='font-size:1em'>l</text>
<text text-anchor='middle' x='192' y='4' fill='currentColor' style='font-size:1em'>o</text>
<text text-anchor='middle' x='192' y='20' fill='currentColor' style='font-size:1em'>g</text>
<text text-anchor='middle' x='200' y='4' fill='currentColor' style='font-size:1em'>c</text>
<text text-anchor='middle' x='200' y='20' fill='currentColor' style='font-size:1em'>r</text>
<text text-anchor='middle' x='208' y='4' fill='currentColor' style='font-size:1em'>a</text>
<text text-anchor='middle' x='208' y='20' fill='currentColor' style='font-size:1em'>o</text>
<text text-anchor='middle' x='216' y='4' fill='currentColor' style='font-size:1em'>l</text>
<text text-anchor='middle' x='216' y='20' fill='currentColor' style='font-size:1em'>u</text>
<text text-anchor='middle' x='224' y='4' fill='currentColor' style='font-size:1em'>/</text>
<text text-anchor='middle' x='224' y='20' fill='currentColor' style='font-size:1em'>p</text>
<text text-anchor='middle' x='232' y='4' fill='currentColor' style='font-size:1em'>a</text>
<text text-anchor='middle' x='232' y='20' fill='currentColor' style='font-size:1em'>s</text>
<text text-anchor='middle' x='240' y='4' fill='currentColor' style='font-size:1em'>p</text>
<text text-anchor='middle' x='240' y='20' fill='currentColor' style='font-size:1em'>=</text>
<text text-anchor='middle' x='248' y='4' fill='currentColor' style='font-size:1em'>a</text>
<text text-anchor='middle' x='248' y='20' fill='currentColor' style='font-size:1em'>0</text>
<text text-anchor='middle' x='256' y='4' fill='currentColor' style='font-size:1em'>c</text>
<text text-anchor='middle' x='256' y='20' fill='currentColor' style='font-size:1em'>(</text>
<text text-anchor='middle' x='264' y='4' fill='currentColor' style='font-size:1em'>h</text>
<text text-anchor='middle' x='264' y='20' fill='currentColor' style='font-size:1em'>r</text>
<text text-anchor='middle' x='272' y='4' fill='currentColor' style='font-size:1em'>e</text>
<text text-anchor='middle' x='272' y='20' fill='currentColor' style='font-size:1em'>o</text>
<text text-anchor='middle' x='280' y='4' fill='currentColor' style='font-size:1em'>2</text>
<text text-anchor='middle' x='280' y='20' fill='currentColor' style='font-size:1em'>o</text>
<text text-anchor='middle' x='288' y='4' fill='currentColor' style='font-size:1em'>#</text>
<text text-anchor='middle' x='288' y='20' fill='currentColor' style='font-size:1em'>t</text>
<text text-anchor='middle' x='296' y='20' fill='currentColor' style='font-size:1em'>)</text>
<text text-anchor='middle' x='304' y='4' fill='currentColor' style='font-size:1em'>i</text>
<text text-anchor='middle' x='312' y='4' fill='currentColor' style='font-size:1em'>d</text>
</g>

    </svg>
  
</div>
<h3 id="process-id-pid">Process ID (<code>PID</code>)</h3>
<p>Each process running on the host has a unique Process ID (PID) assigned to it. The PIDs of processes running inside container are separate from PIDs assigned by the host. Due to process ID isolation, a container can&rsquo;t access the details of processes running on its host.</p>
<p>To fetch the list of PID namespaces you can use the command:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">$ lsns -t pid
</span></span><span class="line"><span class="cl">        NS TYPE NPROCS   PID USER  COMMAND
</span></span><span class="line"><span class="cl"><span class="m">4026531836</span> pid     <span class="m">128</span>  <span class="m">4996</span> user /usr/lib/systemd/systemd --user
</span></span><span class="line"><span class="cl"><span class="m">4026533251</span> pid      <span class="m">16</span>  <span class="m">4525</span> user nginx: worker process
</span></span><span class="line"><span class="cl">...
</span></span></code></pre></div><p>Just like other processes, containers also have PIDs assigned to them by the host. You can fetch the PID of a running container using the following command:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">$  docker inspect -f <span class="s1">&#39;{{.State.Pid}}&#39;</span> deploy-hugo-server-1
</span></span><span class="line"><span class="cl"><span class="m">7172</span>
</span></span></code></pre></div><p><code>ps aux</code> command could be used to list the running processes on the system along with their details.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">$ ps aux <span class="p">|</span> grep <span class="m">7172</span>
</span></span><span class="line"><span class="cl">root        <span class="m">7172</span>  0.0  0.4 <span class="m">759596</span> <span class="m">68860</span> ?        Ssl  Jan22   0:32 /usr/lib/hugo/hugo server --buildFuture --bind<span class="o">=</span>0.0.0.0
</span></span></code></pre></div><h3 id="mount-mnt">Mount (<code>mnt</code>)</h3>
<p>By using different mount namespaces for different processes we can ensure that they won&rsquo;t be able to access each other&rsquo;s files.</p>
<p>You can use <code>df</code> command to view the filesystems mounted on your system.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">$ df
</span></span><span class="line"><span class="cl">Filesystem     1K-blocks      Used Available Use% Mounted on
</span></span><span class="line"><span class="cl">devtmpfs            <span class="m">4096</span>         <span class="m">0</span>      <span class="m">4096</span>   0% /dev
</span></span><span class="line"><span class="cl">tmpfs            <span class="m">7849324</span>     <span class="m">42384</span>   <span class="m">7806940</span>   1% /dev/shm
</span></span><span class="line"><span class="cl">tmpfs            <span class="m">3139732</span>      <span class="m">2444</span>   <span class="m">3137288</span>   1% /run
</span></span><span class="line"><span class="cl">...
</span></span></code></pre></div><p>A container has its separate file system hierarchy which could be viewed by using <code>df</code> command on its shell.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">root@14ed72afd62e:/usr/local/apache2# df
</span></span><span class="line"><span class="cl">Filesystem     1K-blocks      Used Available Use% Mounted on
</span></span><span class="line"><span class="cl">overlay        <span class="m">498443264</span> <span class="m">308159584</span> <span class="m">185270656</span>  63% /
</span></span><span class="line"><span class="cl">tmpfs              <span class="m">65536</span>         <span class="m">0</span>     <span class="m">65536</span>   0% /dev
</span></span><span class="line"><span class="cl">tmpfs            <span class="m">1569860</span>       <span class="m">276</span>   <span class="m">1569584</span>   1% /etc/hosts
</span></span><span class="line"><span class="cl">...
</span></span></code></pre></div><p>It could also be viewed by the host in the file <code>/proc/&lt;CONTAINER_PID&gt;/mounts</code>.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">$ docker inspect -f <span class="s1">&#39;{{.State.Pid}}&#39;</span> deploy-hugo-server-1
</span></span><span class="line"><span class="cl"><span class="m">7172</span>
</span></span><span class="line"><span class="cl">
</span></span><span class="line"><span class="cl">$ cat /proc/7172/mounts
</span></span><span class="line"><span class="cl">proc /proc proc rw,nosuid,nodev,noexec,relatime <span class="m">0</span> <span class="m">0</span>
</span></span><span class="line"><span class="cl">tmpfs /dev tmpfs rw,seclabel,nosuid,size<span class="o">=</span>65536k,mode<span class="o">=</span>755,inode64 <span class="m">0</span> <span class="m">0</span>
</span></span><span class="line"><span class="cl">devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid<span class="o">=</span>5,mode<span class="o">=</span>620,ptmxmode<span class="o">=</span><span class="m">666</span> <span class="m">0</span> <span class="m">0</span>
</span></span><span class="line"><span class="cl">...
</span></span></code></pre></div><h3 id="unix-timesharing-system-uts">Unix Timesharing System (UTS)</h3>
<p>Unix Time System (UTS) namespace allows containers to have hostnames. We can verify this with the <code>hostname</code> command.</p>
<p>On the host:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl"><span class="o">[</span>username@host-1 ~<span class="o">]</span>$ hostname
</span></span><span class="line"><span class="cl">host-1
</span></span></code></pre></div><p>Inside a container:</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">root@14ed72afd62e:/usr/local/apache2# hostname
</span></span><span class="line"><span class="cl">14ed72afd62e
</span></span></code></pre></div><h3 id="network">Network</h3>
<p>Each container has a IP address and network ports assigned to it by its network namespace. It allows the developer to run multiple processes inside the container and expose them over different network ports.</p>
<p>To access or communicate with a process inside the container, port forwarding should be established from the host.</p>
<h3 id="inter-process-communication-ipc">Inter-Process Communication (IPC)</h3>
<p>Processes in the same IPC namespace can share the resources such as memory, semaphores, and message queues. Keeping separate IPC namespaces ensures that the processes inside a container cannot access the resources used by the host&rsquo;s processes.</p>
<h3 id="time">Time</h3>
<p>Time namespaces are available since the release of Linux Kernel 5.6.<br>
Maybe in the future containers can have a different time than their host.</p>
<h2 id="control-groups-cgroups">Control Groups (Cgroups)</h2>
<p>A <strong>control group</strong> is created to effectively allocate resources of host its processes. These Cgroups are hierarchical i.e. a child Cgroup could be spawned from the parent and it will inherit its certain attributes.</p>
<p>By creating a Cgroup a process in it could be prioritized, paused, removed, or resumed based on the resources allocated to it. It also helps in monitoring the resources used by particular processes.</p>
<p>If you are using an OS with <code>systemd</code> init system (to verify this you can use the command <code>ps -p 1 -o comm=</code>) then you can use the command <code>systemctl list-units</code> to list all the Cgroups. It will open a table containing the Cgroup name, state, and description. The names of the Cgroup will be in the form <code>&lt;parent-cgroup&gt;.&lt;child-cgroup&gt;</code> like <code>sys-devices-platform-serial8250-tty-ttyS0.device</code>.</p>
<p>To view the hierarchy of Cgroups you can use the command <code>systemd-cgls</code>. It presents cgroups as a tree structure.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">Control group /:
</span></span><span class="line"><span class="cl">-.slice
</span></span><span class="line"><span class="cl">├─user.slice <span class="o">(</span><span class="c1">#1309)</span>
</span></span><span class="line"><span class="cl">│ └─user-1000.slice <span class="o">(</span><span class="c1">#10010)</span>
</span></span><span class="line"><span class="cl">│   ├─user@1000.service <span class="o">(</span><span class="c1">#10106)</span>
</span></span><span class="line"><span class="cl">│   │ ├─session.slice <span class="o">(</span><span class="c1">#10394)</span>
</span></span><span class="line"><span class="cl">│   │ │ ├─dbus-broker.service <span class="o">(</span><span class="c1">#10442)</span>
</span></span><span class="line"><span class="cl">│   │ │ │ ├─ <span class="m">5088</span> /usr/bin/dbus-broker-launch --scope user
</span></span><span class="line"><span class="cl">...
</span></span></code></pre></div><h2 id="secure-computing-seccomp">Secure Computing (Seccomp)</h2>
<p>Using <strong>Secure Computing</strong> (or seccomp) you can disable the system calls your process can make to the host&rsquo;s kernel.</p>
<p>A <em>seccomp profile</em> is a definition with a set of restricted and allowed system calls stored in a file. Default seccomp profile used by Docker: <a href="https://github.com/moby/moby/blob/master/profiles/seccomp/default.json" target="_blank">default.json</a>.</p>
<p>Docker allows you to define your seccomp profile for a container in JSON format.</p>
<div class="highlight"><pre tabindex="0" class="chroma"><code class="language-bash" data-lang="bash"><span class="line"><span class="cl">docker run --rm -it --security-opt <span class="nv">seccomp</span><span class="o">=</span>/path/to/seccomp/profile.json hello-world
</span></span></code></pre></div><h2 id="security-enhanced-linux-selinux">Security-Enhanced Linux (SELinux)</h2>
<p><strong>SELinux</strong> is a security architecture for GNU/Linux-based OS that defines access to files and processes. It is enforced on users or processes to restrict their access to the resources.</p>
<p>SELinux checks the <em>SELinux context</em> of the file or process to make decisions related to its access control. To view the SELinux context of a file use command <code>ls -Z &lt;FILENAME&gt;</code> and to view it for a process using the command <code>ps -eZ | grep &lt;PROCESS_NAME&gt;</code>.</p>
<h1 id="resources">Resources</h1>
<p><a href="https://www.youtube.com/watch?v=sK5i-N34im8" target="_blank">Cgroups, namespaces, and beyond: what are containers made from?</a><br>
<a href="https://opensource.com/article/21/8/container-linux-technology" target="_blank">4 Linux technologies fundamental to containers</a><br>
<a href="https://man7.org/linux/man-pages/man1/init.1.html" target="_blank">systemd(1) — Linux manual page</a><br>
<a href="https://www.redhat.com/sysadmin/7-linux-namespaces" target="_blank">The 7 most used Linux namespaces</a><br>
<a href="https://opensource.com/article/19/4/interprocess-communication-linux-storage" target="_blank">Inter-process communication in Linux: Shared storage</a><br>
<a href="https://www.tenable.com/audits/items/DISA_STIG_Docker_Enterprise_2.x_Linux_Unix_v1r1.audit:f0aaa2cd4bcb66461902f47c69bf323b" target="_blank">DKER-EE-001250 - The Docker Enterprise hosts IPC namespace must not be shared.</a><br>
<a href="https://www.phoronix.com/news/Time-Namespace-In-Linux-5.6" target="_blank">It&rsquo;s Finally Time: The Time Namespace Support Has Been Added To The Linux 5.6 Kernel</a><br>
<a href="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/resource_management_guide/sec-obtaining_information_about_control_groups" target="_blank">Obtaining Information about Control Groups</a><br>
<a href="https://man7.org/linux/man-pages/man2/seccomp.2.html" target="_blank">seccomp(2) — Linux manual page</a><br>
<a href="https://www.redhat.com/en/topics/linux/what-is-selinux" target="_blank">What is SELinux?</a><br>
<a href="https://blog.christophersmart.com/2021/01/31/podman-volumes-and-selinux/" target="_blank">Podman volumes and SELinux</a></p>
]]></content:encoded>
    </item>
  </channel>
</rss>
