magicpak

Actions Status Actions Status License

magicpak enables you to build minimal docker images without any bothersome preparation such as static linking.

```dockerfile

You prepare /bin/your_executable here...

ADD https://github.com/coord-e/magicpak/releases/download/v1.4.0/magicpak-x86_64-unknown-linux-musl /usr/bin/magicpak RUN chmod +x /usr/bin/magicpak

RUN /usr/bin/magicpak -v /bin/your_executable /bundle

FROM scratch COPY --from=0 /bundle /.

CMD ["/bin/your_executable"] ```

That's it! The resulting image shall only contain what your executable requires at runtime. You can find more useful examples of magicpak under example/.

Feature

magicpak is a command-line utility that analyzes and bundles runtime dependencies of the executable. magicpak basically collects all shared object dependencies that are required by a dynamic linker at runtime. Additionally, magicpak's contributions are summarized as follows:

magicpak is especially useful when you find it difficult to produce a statically linked executable. Also, magicpak is powerful when building from source is bothering or the source code is not public, because magicpak only requires the executable to build a minimal docker image.

Usage

You can start with magicpak path/to/executable path/to/output. This simply analyzes runtime dependencies of your executable statically and put everything your executable needs in runtime to the specified output directory. Once they've bundled, we can simply copy them to the scratch image in the second stage as follows.

```dockerfile RUN magicpak path/to/executable /bundle

FROM scratch COPY --from=0 /bundle /. ```

Some executables work well in this way. However, others fail to run properly because magicpak's static analysis isn't enough to detect all files needed by them at runtime. For this case, magicpak has --include <GLOB> option to specify the missing requirements manually. Moreover, you can use --dynamic to automatically include files that are accessed by the executable during execution.

Despite our careful implementation, our analysis is unreliable in a way because we can't completely determine the runtime behavior before its execution. To ensure that magicpak collected all dependencies to perform a specific task, --test option is implemented. --test enables testing of the resulting bundle using chroot(2).

The size of the resulting image is our main concern. magicpak supports executable compression using upx. You can enable it with --compress.

Supported options

``` Usage: magicpak [OPTIONS] ...

Arguments: ... Input executable Output destination

Options: -i, --include Additionally include files/directories with glob patterns -e, --exclude Exclude files/directories from the resulting bundle with glob patterns --mkdir Make directories in the resulting bundle -r, --install-to Specify the installation path of the executable in the bundle --log-level Specify the log level [default: Warn] [possible values: Off, Error, Warn, Info, Debug] -v, --verbose Verbose mode, same as --log-level Info -t, --test Enable testing --test-command Specify the test command to use in --test --test-stdin Specify stdin content supplied to the test command in --test --test-stdout Test stdout of the test command -d, --dynamic Enable dynamic analysis --dynamic-arg Specify arguments passed to the executable in --dynamic --dynamic-stdin Specify stdin content supplied to the executable in --dynamic -c, --compress Compress the executable with npx --upx-arg Specify arguments passed to upx in --compress --busybox Specify the path or name of busybox that would be used in testing [default: busybox] --upx Specify the path or name of upx that would be used in compression [default: upx] --cc Specify the path or name of c compiler that would be used in the name resolution of shared library dependencies [env: CC=] [default: cc] --experimental-noload-resolver [EXPERIMENTAL] Resolve dynamic library paths without loading in dlopen(3) -h, --help Print help information ```

Docker images

We provide some base images that contain magicpak and its optional dependencies to get started.

| name | description | | ------------------------------------------------------------ | ------------------------------------------------------------ | | magicpak/debian magicpak/debian | library/debian with magicpak | | magicpak/cc magicpak/cc | library/debian with build-essential, clang, and magicpak | | magicpak/haskell magicpak/haskell | library/haskell with magicpak | | magicpak/rust magicpak/rust | library/rust with magicpak |

Example

The following is a dockerfile using magicpak for a docker image of clang-format, a formatter for C/C++/etc. (example/clang-format)

```dockerfile FROM magicpak/debian:buster-magicpak1.4.0

RUN apt-get -y update RUN apt-get -y --no-install-recommends install clang-format

RUN magicpak $(which clang-format) /bundle -v \ --compress \ --upx-arg --best \ --test \ --test-stdin "int main( ){ }" \ --test-stdout "int main() {}" \ --install-to /bin/

FROM scratch COPY --from=0 /bundle /.

WORKDIR /workdir

CMD ["/bin/clang-format"] ```

Note on name resolution and glibc

If your program uses glibc for name resolution (most likely it does), the call to getaddrinfo(3) will result in an error after bundled by magicpak. This can be resolved by manually including the NSS-related shared libraries as shown below.

```dockerfile

example on x86_64 Debian-based image:

RUN magicpak path/to/executable /bundle --include '/lib/x8664-linux-gnu/libnss*' ```

Note on jemalloc

If your program depends on libjemalloc, magicpak may fail with the following message.

error: Unable to lookup shared library: /lib/aarch64-linux-gnu/libjemalloc.so.2: cannot allocate memory in static TLS block

You can use --experimental-noload-resolver flag to workaround this. See #19 for details.

Disclaimer

magicpak comes with absolutely no warranty. There's no guarantee that the processed bundle works properly and identically to the original executable. Although I had no problem using magicpak for building various kinds of images, it is recommended to use this with caution and make a careful examination of the resulting bundle.

License

Licensed under either of

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.