Compiling (La)TeX files automatically with GitLab CI
Inspired by this question about compiling LaTeX files with Travis CI:
How can GitLab's integrated CI feature be used to build (La)TeX documents?
compiling automation tools revision-control continuous-integration
add a comment |
Inspired by this question about compiling LaTeX files with Travis CI:
How can GitLab's integrated CI feature be used to build (La)TeX documents?
compiling automation tools revision-control continuous-integration
1
Would you mind if I changed the title to be a bit more generic ("(La)TeX" instead of "LaTeX")?
– TeXnician
Nov 11 '18 at 16:51
1
No, I wouldn’t mind at all
– JBantje
Nov 11 '18 at 16:54
add a comment |
Inspired by this question about compiling LaTeX files with Travis CI:
How can GitLab's integrated CI feature be used to build (La)TeX documents?
compiling automation tools revision-control continuous-integration
Inspired by this question about compiling LaTeX files with Travis CI:
How can GitLab's integrated CI feature be used to build (La)TeX documents?
compiling automation tools revision-control continuous-integration
compiling automation tools revision-control continuous-integration
edited 13 hours ago
PHPirate
743521
743521
asked Nov 11 '18 at 16:33
JBantjeJBantje
9471513
9471513
1
Would you mind if I changed the title to be a bit more generic ("(La)TeX" instead of "LaTeX")?
– TeXnician
Nov 11 '18 at 16:51
1
No, I wouldn’t mind at all
– JBantje
Nov 11 '18 at 16:54
add a comment |
1
Would you mind if I changed the title to be a bit more generic ("(La)TeX" instead of "LaTeX")?
– TeXnician
Nov 11 '18 at 16:51
1
No, I wouldn’t mind at all
– JBantje
Nov 11 '18 at 16:54
1
1
Would you mind if I changed the title to be a bit more generic ("(La)TeX" instead of "LaTeX")?
– TeXnician
Nov 11 '18 at 16:51
Would you mind if I changed the title to be a bit more generic ("(La)TeX" instead of "LaTeX")?
– TeXnician
Nov 11 '18 at 16:51
1
1
No, I wouldn’t mind at all
– JBantje
Nov 11 '18 at 16:54
No, I wouldn’t mind at all
– JBantje
Nov 11 '18 at 16:54
add a comment |
4 Answers
4
active
oldest
votes
Beware: This is a post about ConTeXt
Apart from LaTeX you may want to compile ConTeXt documents using Gitlab CI. That's very easy as well. Simply use the install script provided by ConTeXt standalone (the following CI configurations will download the beta version of ConTeXt).
Basic example compiling a ConTeXt document:
stages:
- build
build:
stage: build
image: debian:unstable
script:
- apt-get update && apt-get install -y wget rsync unzip && rm -rf /var/lib/apt/lists/*
- mkdir context && cd context
- wget http://minimals.contextgarden.net/setup/first-setup.sh
- sh ./first-setup.sh --modules=all --context=beta --engine=luatex
- . tex/setuptex
- cd ..
- context document.tex
artifacts:
paths:
- ./*.pdf
More sophisticated installing custom fonts (again, I will use Fira Code from the git repo):
stages:
- build
build:
stage: build
image: debian:unstable
script:
- apt-get update && apt-get install -y wget rsync unzip git && rm -rf /var/lib/apt/lists/*
- mkdir context && cd context
- wget http://minimals.contextgarden.net/setup/first-setup.sh
- sh ./first-setup.sh --modules=all --context=beta --engine=luatex
- . tex/setuptex
- cd .. && mkdir fonts && cd fonts
- git clone https://github.com/tonsky/FiraCode firacode
- export OSFONTDIR="/usr/local/share/fonts;$HOME/.fonts;`pwd`"
- mtxrun --script fonts --reload
- cd ..
- context document.tex
artifacts:
paths:
- ./*.pdf
For the sake of completeness: Here's a docker file you might want to use to have a certain ConTeXt version as a container for multiple projects:
FROM alpine:latest
RUN apk update && apk add wget rsync unzip libgcc
RUN mkdir context && cd context && wget http://minimals.contextgarden.net/setup/first-setup.sh &&
sh ./first-setup.sh --modules=all --context=beta --engine=luatex
ENV PATH "/context/tex/texmf-linuxmusl-64/bin:$PATH"
add a comment |
Backstory
I maintain a huge repository of lecture notes written in LaTeX, once hosted on GitHub. Now it can be found HERE on GitLab.com. I use Continuous Integration to automatically compile all relevant .tex
files and upload them to an enterprise-scale OwnCloud installation.
I used a Travis CI setup as in the linked question a few years back.
How does GitLab CI work in general?
If activated in a repo, GitLab can run so called "pipelines" when new commits are pushed. The CI properties are mainly handled by the .gitlab-ci.yml
configuration file, which allows for a ton of customisation: several stages (like building, testing, deployment) may be defined, branches can be handled separately, secret variables, …
This configuration file also specifies, what commands/scripts are to be run in a CI job. But which machine actually runs these scripts? – This is handled by "runners", which are assigned to a repo. Any computer can become a runner by installing the GitLab Runner software, which is available for all major operating systems or even as a Docker container.
The results (console log and maybe artefacts) are sent back to GitLab by the runner. There are public runners operated by GitLab.com, which may be used freely under certain limitations (and aren't very helpful for LaTeX, I assume).
For more details, see the documentation of GitLab.
My Setup for Compiling .tex
files
On my home server I have the runner software installed (which, on Linux, adds a user "gitlab-runner"), which is easy to maintain via apt
. Furthermore I installed TeXlive without the package manager (aka "the recommended way") and made sure that the user "gitlab-runner" can use it.
From the .gitlab-ci.yml
file a makefile
is called, which in turn calls a bash script for the compilation of the several files. Empowered by the caching feature, latexmk
is only run for modified files and only the updated files are reuploaded via WebDAV to the aforementioned OwnCloud installation.
Details and my .gitlab-ci.yml
file can be found in the repo linked above.
Pros and Cons of this GitLab CI solution
(In comparison to the approach with Travis CI and GitHub mentioned earlier)
Pros:
- the CI jobs are executed very fast, since there is no Docker overhead and the runner almost immediately starts the actual compilation
- the caching prevents unnecessary compilation
- a full featured TeXlive installation is used
- imho GitLab is preferable over GitHub for various reasons (self hosted installation etc.)
- Extremely customisable (up to the TeXlive installation)
Cons:
- the need to maintain a server (energy consumption of a home server, hosting fees for a webserver, but theoretically some office PC could do the job as well)
My Verdict
Common CI services like Travis are not perfectly suited for the niche application LaTeX, because of the docker shenanigans needed to get it done – GitLab's integrated CI feature allows for a tailored solution, which is not only faster but more powerful.
If there are further questions feel free to ask.
2
you don't need to use docker to run latex on travis of course you could just use apt-get to get a tex installation in the vm or (as we do for the core latex test suite) get the upstream texlive installer and install a minimal texlive scheme, you can cache the resulting installation directory so there is little overhead after the first time.
– David Carlisle
Nov 11 '18 at 17:24
add a comment |
This answer provides examples for CI files that I use to compile LaTeX documents on Gitlab CI. They use the sumdoc/texlive-2018
image to provide a basic TeX Live installation including Pygmentize. If you want to use one of these, simply put them into your .gitlab-ci.yml
file.
This is a very simple alternative, if you are willing to compile your document using pdflatex or arara.
image: sumdoc/texlive-2018
before_script:
- apt-get update && apt-get install -y openjdk-8-jre && rm -rf /var/lib/apt/lists/*
- tlmgr update --self --all
build:
script:
- arara -lv document.tex
artifacts:
paths:
- ./*.pdf
You may even do fancy font installations (in this case pulling Fira Code using git) and then use them within your document:
image: sumdoc/texlive-2018
before_script:
- apt-get update && apt-get install -y openjdk-8-jre git && rm -rf /var/lib/apt/lists/*
- git clone https://github.com/tonsky/FiraCode firacode && cp firacode/distr/otf/*.otf /usr/local/share/fonts/
- fc-cache -fv
- tlmgr update --self --all
build:
script:
- lualatex document.tex
artifacts:
paths:
- ./*.pdf
If you want a smaller image file have a look at my answer here. That reduces the image size to about 1.7 GiB.
add a comment |
For my cv I use the script below:
compile_pdf:
image: aergus/latex
script:
- latexmk -r .latexmkrc -pdf 'resume.tex' -jobname=resume
artifacts:
paths:
- ./build/resume.pdf
This is saved in a file called .gitlab-ci.yml
and will build a new version every time a new commit is made.
My .latexmkrc
contains the following:
$latex = 'latex %O --shell-escape %S';
$pdflatex = 'lualatex %O --shell-escape %S';
$out_dir = 'build';
I use the --shell-escape command for the minted package.
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "85"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2ftex.stackexchange.com%2fquestions%2f459484%2fcompiling-latex-files-automatically-with-gitlab-ci%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
4 Answers
4
active
oldest
votes
4 Answers
4
active
oldest
votes
active
oldest
votes
active
oldest
votes
Beware: This is a post about ConTeXt
Apart from LaTeX you may want to compile ConTeXt documents using Gitlab CI. That's very easy as well. Simply use the install script provided by ConTeXt standalone (the following CI configurations will download the beta version of ConTeXt).
Basic example compiling a ConTeXt document:
stages:
- build
build:
stage: build
image: debian:unstable
script:
- apt-get update && apt-get install -y wget rsync unzip && rm -rf /var/lib/apt/lists/*
- mkdir context && cd context
- wget http://minimals.contextgarden.net/setup/first-setup.sh
- sh ./first-setup.sh --modules=all --context=beta --engine=luatex
- . tex/setuptex
- cd ..
- context document.tex
artifacts:
paths:
- ./*.pdf
More sophisticated installing custom fonts (again, I will use Fira Code from the git repo):
stages:
- build
build:
stage: build
image: debian:unstable
script:
- apt-get update && apt-get install -y wget rsync unzip git && rm -rf /var/lib/apt/lists/*
- mkdir context && cd context
- wget http://minimals.contextgarden.net/setup/first-setup.sh
- sh ./first-setup.sh --modules=all --context=beta --engine=luatex
- . tex/setuptex
- cd .. && mkdir fonts && cd fonts
- git clone https://github.com/tonsky/FiraCode firacode
- export OSFONTDIR="/usr/local/share/fonts;$HOME/.fonts;`pwd`"
- mtxrun --script fonts --reload
- cd ..
- context document.tex
artifacts:
paths:
- ./*.pdf
For the sake of completeness: Here's a docker file you might want to use to have a certain ConTeXt version as a container for multiple projects:
FROM alpine:latest
RUN apk update && apk add wget rsync unzip libgcc
RUN mkdir context && cd context && wget http://minimals.contextgarden.net/setup/first-setup.sh &&
sh ./first-setup.sh --modules=all --context=beta --engine=luatex
ENV PATH "/context/tex/texmf-linuxmusl-64/bin:$PATH"
add a comment |
Beware: This is a post about ConTeXt
Apart from LaTeX you may want to compile ConTeXt documents using Gitlab CI. That's very easy as well. Simply use the install script provided by ConTeXt standalone (the following CI configurations will download the beta version of ConTeXt).
Basic example compiling a ConTeXt document:
stages:
- build
build:
stage: build
image: debian:unstable
script:
- apt-get update && apt-get install -y wget rsync unzip && rm -rf /var/lib/apt/lists/*
- mkdir context && cd context
- wget http://minimals.contextgarden.net/setup/first-setup.sh
- sh ./first-setup.sh --modules=all --context=beta --engine=luatex
- . tex/setuptex
- cd ..
- context document.tex
artifacts:
paths:
- ./*.pdf
More sophisticated installing custom fonts (again, I will use Fira Code from the git repo):
stages:
- build
build:
stage: build
image: debian:unstable
script:
- apt-get update && apt-get install -y wget rsync unzip git && rm -rf /var/lib/apt/lists/*
- mkdir context && cd context
- wget http://minimals.contextgarden.net/setup/first-setup.sh
- sh ./first-setup.sh --modules=all --context=beta --engine=luatex
- . tex/setuptex
- cd .. && mkdir fonts && cd fonts
- git clone https://github.com/tonsky/FiraCode firacode
- export OSFONTDIR="/usr/local/share/fonts;$HOME/.fonts;`pwd`"
- mtxrun --script fonts --reload
- cd ..
- context document.tex
artifacts:
paths:
- ./*.pdf
For the sake of completeness: Here's a docker file you might want to use to have a certain ConTeXt version as a container for multiple projects:
FROM alpine:latest
RUN apk update && apk add wget rsync unzip libgcc
RUN mkdir context && cd context && wget http://minimals.contextgarden.net/setup/first-setup.sh &&
sh ./first-setup.sh --modules=all --context=beta --engine=luatex
ENV PATH "/context/tex/texmf-linuxmusl-64/bin:$PATH"
add a comment |
Beware: This is a post about ConTeXt
Apart from LaTeX you may want to compile ConTeXt documents using Gitlab CI. That's very easy as well. Simply use the install script provided by ConTeXt standalone (the following CI configurations will download the beta version of ConTeXt).
Basic example compiling a ConTeXt document:
stages:
- build
build:
stage: build
image: debian:unstable
script:
- apt-get update && apt-get install -y wget rsync unzip && rm -rf /var/lib/apt/lists/*
- mkdir context && cd context
- wget http://minimals.contextgarden.net/setup/first-setup.sh
- sh ./first-setup.sh --modules=all --context=beta --engine=luatex
- . tex/setuptex
- cd ..
- context document.tex
artifacts:
paths:
- ./*.pdf
More sophisticated installing custom fonts (again, I will use Fira Code from the git repo):
stages:
- build
build:
stage: build
image: debian:unstable
script:
- apt-get update && apt-get install -y wget rsync unzip git && rm -rf /var/lib/apt/lists/*
- mkdir context && cd context
- wget http://minimals.contextgarden.net/setup/first-setup.sh
- sh ./first-setup.sh --modules=all --context=beta --engine=luatex
- . tex/setuptex
- cd .. && mkdir fonts && cd fonts
- git clone https://github.com/tonsky/FiraCode firacode
- export OSFONTDIR="/usr/local/share/fonts;$HOME/.fonts;`pwd`"
- mtxrun --script fonts --reload
- cd ..
- context document.tex
artifacts:
paths:
- ./*.pdf
For the sake of completeness: Here's a docker file you might want to use to have a certain ConTeXt version as a container for multiple projects:
FROM alpine:latest
RUN apk update && apk add wget rsync unzip libgcc
RUN mkdir context && cd context && wget http://minimals.contextgarden.net/setup/first-setup.sh &&
sh ./first-setup.sh --modules=all --context=beta --engine=luatex
ENV PATH "/context/tex/texmf-linuxmusl-64/bin:$PATH"
Beware: This is a post about ConTeXt
Apart from LaTeX you may want to compile ConTeXt documents using Gitlab CI. That's very easy as well. Simply use the install script provided by ConTeXt standalone (the following CI configurations will download the beta version of ConTeXt).
Basic example compiling a ConTeXt document:
stages:
- build
build:
stage: build
image: debian:unstable
script:
- apt-get update && apt-get install -y wget rsync unzip && rm -rf /var/lib/apt/lists/*
- mkdir context && cd context
- wget http://minimals.contextgarden.net/setup/first-setup.sh
- sh ./first-setup.sh --modules=all --context=beta --engine=luatex
- . tex/setuptex
- cd ..
- context document.tex
artifacts:
paths:
- ./*.pdf
More sophisticated installing custom fonts (again, I will use Fira Code from the git repo):
stages:
- build
build:
stage: build
image: debian:unstable
script:
- apt-get update && apt-get install -y wget rsync unzip git && rm -rf /var/lib/apt/lists/*
- mkdir context && cd context
- wget http://minimals.contextgarden.net/setup/first-setup.sh
- sh ./first-setup.sh --modules=all --context=beta --engine=luatex
- . tex/setuptex
- cd .. && mkdir fonts && cd fonts
- git clone https://github.com/tonsky/FiraCode firacode
- export OSFONTDIR="/usr/local/share/fonts;$HOME/.fonts;`pwd`"
- mtxrun --script fonts --reload
- cd ..
- context document.tex
artifacts:
paths:
- ./*.pdf
For the sake of completeness: Here's a docker file you might want to use to have a certain ConTeXt version as a container for multiple projects:
FROM alpine:latest
RUN apk update && apk add wget rsync unzip libgcc
RUN mkdir context && cd context && wget http://minimals.contextgarden.net/setup/first-setup.sh &&
sh ./first-setup.sh --modules=all --context=beta --engine=luatex
ENV PATH "/context/tex/texmf-linuxmusl-64/bin:$PATH"
edited Nov 11 '18 at 16:56
answered Nov 11 '18 at 16:50
TeXnicianTeXnician
24.7k63187
24.7k63187
add a comment |
add a comment |
Backstory
I maintain a huge repository of lecture notes written in LaTeX, once hosted on GitHub. Now it can be found HERE on GitLab.com. I use Continuous Integration to automatically compile all relevant .tex
files and upload them to an enterprise-scale OwnCloud installation.
I used a Travis CI setup as in the linked question a few years back.
How does GitLab CI work in general?
If activated in a repo, GitLab can run so called "pipelines" when new commits are pushed. The CI properties are mainly handled by the .gitlab-ci.yml
configuration file, which allows for a ton of customisation: several stages (like building, testing, deployment) may be defined, branches can be handled separately, secret variables, …
This configuration file also specifies, what commands/scripts are to be run in a CI job. But which machine actually runs these scripts? – This is handled by "runners", which are assigned to a repo. Any computer can become a runner by installing the GitLab Runner software, which is available for all major operating systems or even as a Docker container.
The results (console log and maybe artefacts) are sent back to GitLab by the runner. There are public runners operated by GitLab.com, which may be used freely under certain limitations (and aren't very helpful for LaTeX, I assume).
For more details, see the documentation of GitLab.
My Setup for Compiling .tex
files
On my home server I have the runner software installed (which, on Linux, adds a user "gitlab-runner"), which is easy to maintain via apt
. Furthermore I installed TeXlive without the package manager (aka "the recommended way") and made sure that the user "gitlab-runner" can use it.
From the .gitlab-ci.yml
file a makefile
is called, which in turn calls a bash script for the compilation of the several files. Empowered by the caching feature, latexmk
is only run for modified files and only the updated files are reuploaded via WebDAV to the aforementioned OwnCloud installation.
Details and my .gitlab-ci.yml
file can be found in the repo linked above.
Pros and Cons of this GitLab CI solution
(In comparison to the approach with Travis CI and GitHub mentioned earlier)
Pros:
- the CI jobs are executed very fast, since there is no Docker overhead and the runner almost immediately starts the actual compilation
- the caching prevents unnecessary compilation
- a full featured TeXlive installation is used
- imho GitLab is preferable over GitHub for various reasons (self hosted installation etc.)
- Extremely customisable (up to the TeXlive installation)
Cons:
- the need to maintain a server (energy consumption of a home server, hosting fees for a webserver, but theoretically some office PC could do the job as well)
My Verdict
Common CI services like Travis are not perfectly suited for the niche application LaTeX, because of the docker shenanigans needed to get it done – GitLab's integrated CI feature allows for a tailored solution, which is not only faster but more powerful.
If there are further questions feel free to ask.
2
you don't need to use docker to run latex on travis of course you could just use apt-get to get a tex installation in the vm or (as we do for the core latex test suite) get the upstream texlive installer and install a minimal texlive scheme, you can cache the resulting installation directory so there is little overhead after the first time.
– David Carlisle
Nov 11 '18 at 17:24
add a comment |
Backstory
I maintain a huge repository of lecture notes written in LaTeX, once hosted on GitHub. Now it can be found HERE on GitLab.com. I use Continuous Integration to automatically compile all relevant .tex
files and upload them to an enterprise-scale OwnCloud installation.
I used a Travis CI setup as in the linked question a few years back.
How does GitLab CI work in general?
If activated in a repo, GitLab can run so called "pipelines" when new commits are pushed. The CI properties are mainly handled by the .gitlab-ci.yml
configuration file, which allows for a ton of customisation: several stages (like building, testing, deployment) may be defined, branches can be handled separately, secret variables, …
This configuration file also specifies, what commands/scripts are to be run in a CI job. But which machine actually runs these scripts? – This is handled by "runners", which are assigned to a repo. Any computer can become a runner by installing the GitLab Runner software, which is available for all major operating systems or even as a Docker container.
The results (console log and maybe artefacts) are sent back to GitLab by the runner. There are public runners operated by GitLab.com, which may be used freely under certain limitations (and aren't very helpful for LaTeX, I assume).
For more details, see the documentation of GitLab.
My Setup for Compiling .tex
files
On my home server I have the runner software installed (which, on Linux, adds a user "gitlab-runner"), which is easy to maintain via apt
. Furthermore I installed TeXlive without the package manager (aka "the recommended way") and made sure that the user "gitlab-runner" can use it.
From the .gitlab-ci.yml
file a makefile
is called, which in turn calls a bash script for the compilation of the several files. Empowered by the caching feature, latexmk
is only run for modified files and only the updated files are reuploaded via WebDAV to the aforementioned OwnCloud installation.
Details and my .gitlab-ci.yml
file can be found in the repo linked above.
Pros and Cons of this GitLab CI solution
(In comparison to the approach with Travis CI and GitHub mentioned earlier)
Pros:
- the CI jobs are executed very fast, since there is no Docker overhead and the runner almost immediately starts the actual compilation
- the caching prevents unnecessary compilation
- a full featured TeXlive installation is used
- imho GitLab is preferable over GitHub for various reasons (self hosted installation etc.)
- Extremely customisable (up to the TeXlive installation)
Cons:
- the need to maintain a server (energy consumption of a home server, hosting fees for a webserver, but theoretically some office PC could do the job as well)
My Verdict
Common CI services like Travis are not perfectly suited for the niche application LaTeX, because of the docker shenanigans needed to get it done – GitLab's integrated CI feature allows for a tailored solution, which is not only faster but more powerful.
If there are further questions feel free to ask.
2
you don't need to use docker to run latex on travis of course you could just use apt-get to get a tex installation in the vm or (as we do for the core latex test suite) get the upstream texlive installer and install a minimal texlive scheme, you can cache the resulting installation directory so there is little overhead after the first time.
– David Carlisle
Nov 11 '18 at 17:24
add a comment |
Backstory
I maintain a huge repository of lecture notes written in LaTeX, once hosted on GitHub. Now it can be found HERE on GitLab.com. I use Continuous Integration to automatically compile all relevant .tex
files and upload them to an enterprise-scale OwnCloud installation.
I used a Travis CI setup as in the linked question a few years back.
How does GitLab CI work in general?
If activated in a repo, GitLab can run so called "pipelines" when new commits are pushed. The CI properties are mainly handled by the .gitlab-ci.yml
configuration file, which allows for a ton of customisation: several stages (like building, testing, deployment) may be defined, branches can be handled separately, secret variables, …
This configuration file also specifies, what commands/scripts are to be run in a CI job. But which machine actually runs these scripts? – This is handled by "runners", which are assigned to a repo. Any computer can become a runner by installing the GitLab Runner software, which is available for all major operating systems or even as a Docker container.
The results (console log and maybe artefacts) are sent back to GitLab by the runner. There are public runners operated by GitLab.com, which may be used freely under certain limitations (and aren't very helpful for LaTeX, I assume).
For more details, see the documentation of GitLab.
My Setup for Compiling .tex
files
On my home server I have the runner software installed (which, on Linux, adds a user "gitlab-runner"), which is easy to maintain via apt
. Furthermore I installed TeXlive without the package manager (aka "the recommended way") and made sure that the user "gitlab-runner" can use it.
From the .gitlab-ci.yml
file a makefile
is called, which in turn calls a bash script for the compilation of the several files. Empowered by the caching feature, latexmk
is only run for modified files and only the updated files are reuploaded via WebDAV to the aforementioned OwnCloud installation.
Details and my .gitlab-ci.yml
file can be found in the repo linked above.
Pros and Cons of this GitLab CI solution
(In comparison to the approach with Travis CI and GitHub mentioned earlier)
Pros:
- the CI jobs are executed very fast, since there is no Docker overhead and the runner almost immediately starts the actual compilation
- the caching prevents unnecessary compilation
- a full featured TeXlive installation is used
- imho GitLab is preferable over GitHub for various reasons (self hosted installation etc.)
- Extremely customisable (up to the TeXlive installation)
Cons:
- the need to maintain a server (energy consumption of a home server, hosting fees for a webserver, but theoretically some office PC could do the job as well)
My Verdict
Common CI services like Travis are not perfectly suited for the niche application LaTeX, because of the docker shenanigans needed to get it done – GitLab's integrated CI feature allows for a tailored solution, which is not only faster but more powerful.
If there are further questions feel free to ask.
Backstory
I maintain a huge repository of lecture notes written in LaTeX, once hosted on GitHub. Now it can be found HERE on GitLab.com. I use Continuous Integration to automatically compile all relevant .tex
files and upload them to an enterprise-scale OwnCloud installation.
I used a Travis CI setup as in the linked question a few years back.
How does GitLab CI work in general?
If activated in a repo, GitLab can run so called "pipelines" when new commits are pushed. The CI properties are mainly handled by the .gitlab-ci.yml
configuration file, which allows for a ton of customisation: several stages (like building, testing, deployment) may be defined, branches can be handled separately, secret variables, …
This configuration file also specifies, what commands/scripts are to be run in a CI job. But which machine actually runs these scripts? – This is handled by "runners", which are assigned to a repo. Any computer can become a runner by installing the GitLab Runner software, which is available for all major operating systems or even as a Docker container.
The results (console log and maybe artefacts) are sent back to GitLab by the runner. There are public runners operated by GitLab.com, which may be used freely under certain limitations (and aren't very helpful for LaTeX, I assume).
For more details, see the documentation of GitLab.
My Setup for Compiling .tex
files
On my home server I have the runner software installed (which, on Linux, adds a user "gitlab-runner"), which is easy to maintain via apt
. Furthermore I installed TeXlive without the package manager (aka "the recommended way") and made sure that the user "gitlab-runner" can use it.
From the .gitlab-ci.yml
file a makefile
is called, which in turn calls a bash script for the compilation of the several files. Empowered by the caching feature, latexmk
is only run for modified files and only the updated files are reuploaded via WebDAV to the aforementioned OwnCloud installation.
Details and my .gitlab-ci.yml
file can be found in the repo linked above.
Pros and Cons of this GitLab CI solution
(In comparison to the approach with Travis CI and GitHub mentioned earlier)
Pros:
- the CI jobs are executed very fast, since there is no Docker overhead and the runner almost immediately starts the actual compilation
- the caching prevents unnecessary compilation
- a full featured TeXlive installation is used
- imho GitLab is preferable over GitHub for various reasons (self hosted installation etc.)
- Extremely customisable (up to the TeXlive installation)
Cons:
- the need to maintain a server (energy consumption of a home server, hosting fees for a webserver, but theoretically some office PC could do the job as well)
My Verdict
Common CI services like Travis are not perfectly suited for the niche application LaTeX, because of the docker shenanigans needed to get it done – GitLab's integrated CI feature allows for a tailored solution, which is not only faster but more powerful.
If there are further questions feel free to ask.
answered Nov 11 '18 at 16:33
JBantjeJBantje
9471513
9471513
2
you don't need to use docker to run latex on travis of course you could just use apt-get to get a tex installation in the vm or (as we do for the core latex test suite) get the upstream texlive installer and install a minimal texlive scheme, you can cache the resulting installation directory so there is little overhead after the first time.
– David Carlisle
Nov 11 '18 at 17:24
add a comment |
2
you don't need to use docker to run latex on travis of course you could just use apt-get to get a tex installation in the vm or (as we do for the core latex test suite) get the upstream texlive installer and install a minimal texlive scheme, you can cache the resulting installation directory so there is little overhead after the first time.
– David Carlisle
Nov 11 '18 at 17:24
2
2
you don't need to use docker to run latex on travis of course you could just use apt-get to get a tex installation in the vm or (as we do for the core latex test suite) get the upstream texlive installer and install a minimal texlive scheme, you can cache the resulting installation directory so there is little overhead after the first time.
– David Carlisle
Nov 11 '18 at 17:24
you don't need to use docker to run latex on travis of course you could just use apt-get to get a tex installation in the vm or (as we do for the core latex test suite) get the upstream texlive installer and install a minimal texlive scheme, you can cache the resulting installation directory so there is little overhead after the first time.
– David Carlisle
Nov 11 '18 at 17:24
add a comment |
This answer provides examples for CI files that I use to compile LaTeX documents on Gitlab CI. They use the sumdoc/texlive-2018
image to provide a basic TeX Live installation including Pygmentize. If you want to use one of these, simply put them into your .gitlab-ci.yml
file.
This is a very simple alternative, if you are willing to compile your document using pdflatex or arara.
image: sumdoc/texlive-2018
before_script:
- apt-get update && apt-get install -y openjdk-8-jre && rm -rf /var/lib/apt/lists/*
- tlmgr update --self --all
build:
script:
- arara -lv document.tex
artifacts:
paths:
- ./*.pdf
You may even do fancy font installations (in this case pulling Fira Code using git) and then use them within your document:
image: sumdoc/texlive-2018
before_script:
- apt-get update && apt-get install -y openjdk-8-jre git && rm -rf /var/lib/apt/lists/*
- git clone https://github.com/tonsky/FiraCode firacode && cp firacode/distr/otf/*.otf /usr/local/share/fonts/
- fc-cache -fv
- tlmgr update --self --all
build:
script:
- lualatex document.tex
artifacts:
paths:
- ./*.pdf
If you want a smaller image file have a look at my answer here. That reduces the image size to about 1.7 GiB.
add a comment |
This answer provides examples for CI files that I use to compile LaTeX documents on Gitlab CI. They use the sumdoc/texlive-2018
image to provide a basic TeX Live installation including Pygmentize. If you want to use one of these, simply put them into your .gitlab-ci.yml
file.
This is a very simple alternative, if you are willing to compile your document using pdflatex or arara.
image: sumdoc/texlive-2018
before_script:
- apt-get update && apt-get install -y openjdk-8-jre && rm -rf /var/lib/apt/lists/*
- tlmgr update --self --all
build:
script:
- arara -lv document.tex
artifacts:
paths:
- ./*.pdf
You may even do fancy font installations (in this case pulling Fira Code using git) and then use them within your document:
image: sumdoc/texlive-2018
before_script:
- apt-get update && apt-get install -y openjdk-8-jre git && rm -rf /var/lib/apt/lists/*
- git clone https://github.com/tonsky/FiraCode firacode && cp firacode/distr/otf/*.otf /usr/local/share/fonts/
- fc-cache -fv
- tlmgr update --self --all
build:
script:
- lualatex document.tex
artifacts:
paths:
- ./*.pdf
If you want a smaller image file have a look at my answer here. That reduces the image size to about 1.7 GiB.
add a comment |
This answer provides examples for CI files that I use to compile LaTeX documents on Gitlab CI. They use the sumdoc/texlive-2018
image to provide a basic TeX Live installation including Pygmentize. If you want to use one of these, simply put them into your .gitlab-ci.yml
file.
This is a very simple alternative, if you are willing to compile your document using pdflatex or arara.
image: sumdoc/texlive-2018
before_script:
- apt-get update && apt-get install -y openjdk-8-jre && rm -rf /var/lib/apt/lists/*
- tlmgr update --self --all
build:
script:
- arara -lv document.tex
artifacts:
paths:
- ./*.pdf
You may even do fancy font installations (in this case pulling Fira Code using git) and then use them within your document:
image: sumdoc/texlive-2018
before_script:
- apt-get update && apt-get install -y openjdk-8-jre git && rm -rf /var/lib/apt/lists/*
- git clone https://github.com/tonsky/FiraCode firacode && cp firacode/distr/otf/*.otf /usr/local/share/fonts/
- fc-cache -fv
- tlmgr update --self --all
build:
script:
- lualatex document.tex
artifacts:
paths:
- ./*.pdf
If you want a smaller image file have a look at my answer here. That reduces the image size to about 1.7 GiB.
This answer provides examples for CI files that I use to compile LaTeX documents on Gitlab CI. They use the sumdoc/texlive-2018
image to provide a basic TeX Live installation including Pygmentize. If you want to use one of these, simply put them into your .gitlab-ci.yml
file.
This is a very simple alternative, if you are willing to compile your document using pdflatex or arara.
image: sumdoc/texlive-2018
before_script:
- apt-get update && apt-get install -y openjdk-8-jre && rm -rf /var/lib/apt/lists/*
- tlmgr update --self --all
build:
script:
- arara -lv document.tex
artifacts:
paths:
- ./*.pdf
You may even do fancy font installations (in this case pulling Fira Code using git) and then use them within your document:
image: sumdoc/texlive-2018
before_script:
- apt-get update && apt-get install -y openjdk-8-jre git && rm -rf /var/lib/apt/lists/*
- git clone https://github.com/tonsky/FiraCode firacode && cp firacode/distr/otf/*.otf /usr/local/share/fonts/
- fc-cache -fv
- tlmgr update --self --all
build:
script:
- lualatex document.tex
artifacts:
paths:
- ./*.pdf
If you want a smaller image file have a look at my answer here. That reduces the image size to about 1.7 GiB.
edited Jan 9 at 9:03
answered Nov 11 '18 at 16:42
TeXnicianTeXnician
24.7k63187
24.7k63187
add a comment |
add a comment |
For my cv I use the script below:
compile_pdf:
image: aergus/latex
script:
- latexmk -r .latexmkrc -pdf 'resume.tex' -jobname=resume
artifacts:
paths:
- ./build/resume.pdf
This is saved in a file called .gitlab-ci.yml
and will build a new version every time a new commit is made.
My .latexmkrc
contains the following:
$latex = 'latex %O --shell-escape %S';
$pdflatex = 'lualatex %O --shell-escape %S';
$out_dir = 'build';
I use the --shell-escape command for the minted package.
add a comment |
For my cv I use the script below:
compile_pdf:
image: aergus/latex
script:
- latexmk -r .latexmkrc -pdf 'resume.tex' -jobname=resume
artifacts:
paths:
- ./build/resume.pdf
This is saved in a file called .gitlab-ci.yml
and will build a new version every time a new commit is made.
My .latexmkrc
contains the following:
$latex = 'latex %O --shell-escape %S';
$pdflatex = 'lualatex %O --shell-escape %S';
$out_dir = 'build';
I use the --shell-escape command for the minted package.
add a comment |
For my cv I use the script below:
compile_pdf:
image: aergus/latex
script:
- latexmk -r .latexmkrc -pdf 'resume.tex' -jobname=resume
artifacts:
paths:
- ./build/resume.pdf
This is saved in a file called .gitlab-ci.yml
and will build a new version every time a new commit is made.
My .latexmkrc
contains the following:
$latex = 'latex %O --shell-escape %S';
$pdflatex = 'lualatex %O --shell-escape %S';
$out_dir = 'build';
I use the --shell-escape command for the minted package.
For my cv I use the script below:
compile_pdf:
image: aergus/latex
script:
- latexmk -r .latexmkrc -pdf 'resume.tex' -jobname=resume
artifacts:
paths:
- ./build/resume.pdf
This is saved in a file called .gitlab-ci.yml
and will build a new version every time a new commit is made.
My .latexmkrc
contains the following:
$latex = 'latex %O --shell-escape %S';
$pdflatex = 'lualatex %O --shell-escape %S';
$out_dir = 'build';
I use the --shell-escape command for the minted package.
answered Nov 12 '18 at 6:23
PascalPascal
363313
363313
add a comment |
add a comment |
Thanks for contributing an answer to TeX - LaTeX Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2ftex.stackexchange.com%2fquestions%2f459484%2fcompiling-latex-files-automatically-with-gitlab-ci%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Would you mind if I changed the title to be a bit more generic ("(La)TeX" instead of "LaTeX")?
– TeXnician
Nov 11 '18 at 16:51
1
No, I wouldn’t mind at all
– JBantje
Nov 11 '18 at 16:54