Documentation

Documentation for jinny, an open source jinja2 CLI tool

Jinny is a templating tool for jinja templates. It can be used for a number of things but was created from a DevOps perspective to aid in configuration management for scaled deployments instead of using tools like Helm, Kustomize, jinja-cli, etc. These days jinny is still used for Ops work but is also used for live applications handling email templating, static HTML generation and more. This very documentation site is built and templated with Jinny. Jinny compiles all the partial templates, the configuration and the CSS into a single HTML file, both during live development and for production artefacts.

Usage

Use Cases

Use cases of individual functionality are provided throughout this documentation. You can find the source code for this site at docs/*. At a high level jinny has been used for: - A lightweight templating tool for Kubernetes manifests - Templating environment variable files for use with docker images - Email & HTTP templating within AWS Lambda functions - Local development templating for static front-ends

Arguments

Jinny takes the following CLI arguments:


usage: jinny [-h] [-v] [-vvv] [-i [INPUTS ...]] [-e [EXPLICIT ...]] -t [TEMPLATES ...] [-ie] [-ds DICT_SEPARATOR] [--version] [--j-block-start J_BLOCK_START] [--j-block-end J_BLOCK_END]
            [--j-variable-start J_VARIABLE_START] [--j-variable-end J_VARIABLE_END] [--j-comment-start J_COMMENT_START] [--j-comment-end J_COMMENT_END] [--j-trim-blocks] [--j-lstrip-blocks]
            [--j-newline-sequence J_NEWLINE_SEQUENCE] [--j-keep-trailing-newline] [-d DUMP_TO_DIR] [-di DUMP_TO_DIR_NO_INDEX] [-s STDOUT_SEPERATOR] [-c] [-ld LOG_DESTINATION] [-nc] [-he]

jinny v1.10.0 | jinny.scripted.dog
Jinny handles complex templating for jinja templates at a large scale and with multiple inputs and with a decent amount of customisation available.

Commonly you'll want to utilse very straight forward features, such as:

=> Templating multiple templates with a single input file:
$ jinny -t template-1.txt template-2.txt -i inputs.yml

=> Templating any number of templates with two input files where base-values.yml provides all the base values and any values in overrides.json acts as an override:
$ jinny -t template.yml -i base-values.yml overrides.json

=> Add even more overrides via environment variables, so your pipelines can completely replace any bad value:
$ JINNY_overridden_value="top-priority" jinny -t template.yml -i base-values.yml overrides.json

=> Or via CLI:
$ jinny -t template.yml -i base-values.yml -e overridden_value="top-priority" overrides.json

=> Pump all your files to a single stdout stream with a separator so different files are clearly marked:
$ jinny -t template-1.yml template-2.yml -i inputs.json -s='---'

=> Dump all your templated files into a directory for capture
$ jinny -t template-1.yml template-2.yml -i inputes.json -d /path/to/directory
$ kubectl diff -f /path/to/directory
$ kubectl apply --server-dry-run -f /path/to/directory

=> Pipe jinny to kubectl for appropriate templating without having to result to Helm
$ jinny -t template-1.yml -i inputs.json | kubectl apply -f -

You can modify jinja's environment settings via the rest of the command line options. Please note that jinny is opinionated and automatically strips line space from templates. You can, of course, turn this off!

options:
  -h, --help            show this help message and exit
  -v, --verbose         Set output to verbose
  -vvv, --super-verbose
                        Set output to super verbose where this script will print basically everything, INCLUDING POTENTIALLY SENSITIVE THINGS!
  -i [INPUTS ...], --inputs [INPUTS ...]
                        Add one or more file locations that include input values to the templating
  -e [EXPLICIT ...], --explicit [EXPLICIT ...]
                        Explicitly define a variable that trumps all other variables using a variable=value format. Adding variables like this trumps every other setting for that variable
  -t [TEMPLATES ...], --templates [TEMPLATES ...]
                        Add one or more file locations that contain the templates themselves
  -ie, --ignore-env-vars
                        Tell jinny to ignore any environment variables that begin with JINNY_, defaults to not ignoring these environment variables and setting them at the highest priority
  -ds DICT_SEPARATOR, --dict-separator DICT_SEPARATOR
                        When providing targeting on the CLI or via environment variables, choose a particular separating character for targeting nested elements, defaults to '.'
  --version             show program's version number and exit
  --j-block-start J_BLOCK_START
                        Change the characters that indicate the start of a block, default '{%'
  --j-block-end J_BLOCK_END
                        Change the characters that indicate the end of a block, default '%}'
  --j-variable-start J_VARIABLE_START
                        Change the characters that indicate the start of a variable, default '{{'
  --j-variable-end J_VARIABLE_END
                        Change the characters that indicate the end of a variable, default '}}'
  --j-comment-start J_COMMENT_START
                        Change the characters that indicate the start of a comment inside of a template, default '{#'
  --j-comment-end J_COMMENT_END
                        Change the characters that indicate the end of a comment within a template, default '#}'
  --j-trim-blocks       Set blocks to trim the newline after a block, this defaults to TRUE in jinny
  --j-lstrip-blocks     Set blocks to trim the whitespace before a block, this defaults to TRUE in jinny
  --j-newline-sequence J_NEWLINE_SEQUENCE
                        This details the newline in use, defaults to \n
  --j-keep-trailing-newline
                        Choose whether to trim the newline at the end of a file or not, defaults to TRUE in jinny
  -d DUMP_TO_DIR, --dump-to-dir DUMP_TO_DIR
                        Dump completed templates to a target directory
  -di DUMP_TO_DIR_NO_INDEX, --dump-to-dir-no-index DUMP_TO_DIR_NO_INDEX
                        Dump completed templates to a target directory without index separation, meaning that templates with the same name can overwrite prior templates
  -s STDOUT_SEPERATOR, --stdout-seperator STDOUT_SEPERATOR
                        Place a seperator on it's own individual new line between successfully templated template when printing to stdout, eg '---' for yaml
  -c, --combine-lists   When cascading values across multiple files and encountering two lists with the same key, choose to combine the old list with the new list rather than have the new list replace the old
  -ld LOG_DESTINATION, --log-destination LOG_DESTINATION
                        Chose an alternate destination to log to, jinny defaults to stdout but you can provide a file to print output to instead
  -nc, --no-color, --no-colour
                        Turn off coloured output
  -he, --html-error     When encountering an error on the current template render a HTML error page with details on the error as well as log the error. This allows for templating errors to be captured by live browser reloads. Seriously, don't use this in prod

CLI Usage Examples

As a CLI tool there are a number of options and CLI arguments that can be provided to jinny. Some common examples are below: 


=> Templating multiple templates with a single input file:
$ jinny -t template-1.txt template-2.txt -i inputs.yml

=> Templating any number of templates with two input files where base-values.yml provides all the base values and any values in overrides.json acts as an override:
$ jinny -t template.yml -i base-values.yml overrides.json

=> Use an environment value file for templating
$ jinny -t template.yml -i ENVIRONMENT_VARIABLES.env

=> Add an explicit override via CLI argument -e
$ jinny -t template.yml -i base-values.yml -e 'image=smasherofallthings/flask-waitress:latest'

=> Add even more overrides via environment variables, so your pipelines can completely replace any bad value:
$ JINNY_overridden_value="top-priority" jinny -t template.yml -i base-values.yml overrides.json

=> Pump all your files to a single stdout stream with a separator so different files are clearly marked:
$ jinny -t template-1.yml template-2.yml -i inputs.json -s '---'

=> Dump all your templated files into a directory for capture, comparison and deployment
$ jinny -t template-1.yml template-2.yml -i inputs.json -d /path/to/directory
$ kubectl diff -f /path/to/directory
$ kubectl apply --server-dry-run -f /path/to/directory

=> Pipe jinny to kubectl directly
$ jinny -t template-1.yml -i inputs.json | kubectl apply -f -

Module Usage Examples

Jinny is primarily a CLI tool, and python packaging can get incredibly painful, however, for simple use cases Jinny still works as a module and has been used in production environments for things like email templating (ie, data from a contact POST request => values for HTML email template to send to internal teams) 


# Import all helpers and standard elements of Jinny
import jinny.jinny as jinny

# If you need Jinny's unsafe functionality then call the ActivateJinnyUnsafe function
jinny.ActivateJinnyUnsafe()

# If you need the custom filters which is all the custom filters and extensions documented below then make sure that you load them in as well
# These are loaded in by default via the CLI route but you can opt into them as a module by running the LoadCustomFilters() function
# You have to activate whether Jinny is loading in as unsafe or not before you load in the custom filters
jinny.LoadCustomFilters()

# And then use it as in this production example when working with a raw string
# When working with raw strings you're not going to have a path to work off of, so some of the path globals won't populate
tmpl = jinny.TemplateHandler(templateName="email", rawString=tmplData)
tmpl.Render({
  'rows': parsedBody
})

# Alternatively provide a path and Jinny will read the file and populate the path as usual
import os
scriptDir = os.path.dirname(__file__)
reportRoot = os.path.join(scriptDir, 'report.html')
tmpl = jinny.TemplateHandler(templateName="report", path=reportRoot)

# The result after the call to Render is in the template object at the result property
print(tmpl.result)


# Values can be exactly defined as the above or you can use the helper functions to give you the same features 
mergedDict = jinny.CombineValues({
  "first": True
}, {
  "second": "please"
}, 'test_merging_dicts')

assert mergedDict["first"] == True
assert mergedDict["second"] == "please"

mergedDict["rows"] = []

jinny.SetNestedValue(mergedDict, ["rows", "0"], "firstItem")

tmpl = jinny.TemplateHandler(templateName="email-two", rawString=tmplData)
tmpl.Render(mergedDict)

print(tmpl.result)

Enhancements

Jinny is opinionated. This means that it makes choices and provides functionality outside of the standard Jinja toolset. One opinion includes automatically trimming template white-space leaving outputs easier to read. Additionally, the below filters and global functions have been added to improve jinny's usefulness out in the wild.

HTML Error Pages

Jinny works beautifully as a templating tool for static HTML. This documentation page is built solely in Jinny. Applications across the globe use Jinny for simple HTML templating providing a framework around dynamic applications or for mass templating of common pages. Working with static HTML can be tedious however, with frontend tooling such as browsersync and nodemon it's possible to have template changes automatically compile via Jinny and have your local browser refresh automatically with the updated output. When something goes wrong with your templating it may not be obvious that a templating error has occured, therefore Jinny has the `--html-error` or -he option which will generate a HTML error page in place of an errored template. For example, the below error page is produced when the template at `/home/smashthings/nonsense.html` references an undefined variable `somevariable`.

Filters

Filters take passed in content via jinja pipes so tend to be used for chaining together and manipulating text provided to them.

file_content

Fully imports the raw content of a file into the template where called.


$ cat template.html

<html>
<style>
{{ ( path.templatedir + "css.css" ) | file_content }}
</style>
</html>

$ cat css.css

html { font-weight: 900; }

$ jinny -t template.html
<html>
<style>
html { font-weight: 900; }
</style>
</html>

nested_template

Imports and templates other templates with the same values as the master template has received. This may not be thread safe for non-GIL or other multi-threaded implementations of python as it relies on pointer updates to a global variable from the root template. For CPython which is the vast majority of implementations and run-time this is totally fine. The benefit of this approach is dodging passing the value stack through the Jinja codebase which can be prone to breaking in Jinja updates and adds a substantial overhead to route and debug. This functionality is well outside of the Jinja's intended design. If you don't want to deviate too far from Jinja then don't use it. 


$ cat template.html

<html>
<style>
{{ ( path.templatedir + "css.css" ) | nested_template }}
</style>
</html>

$ cat css.css

html { font-weight: {{ font_weight }}; }

$ jinny -t template.html -e font_weight=600
<html>
<style>
html { font-weight: 600; }
</style>
</html>

print_stdout, print_stderr & tee

These filters will print to stdout, stderr or tee to stdout and continue to content. They're used for debugging, warnings and other elements. You are going to want to run this with `-d` or `-di` options so that resulting templates are written to files rather than dumped to standard out. Not doing that will mix the output of these filters into your template output. With that caveat in mind this allows for template annotations within the templating process telling you what's going on without having to analyse the output: 


$ cat template.html

<html>
<style>
  {{ ("Running a build of this template " + path.template + " at: ") | print_stdout }}
  <h1> This page was generated at {{ time_now() | tee }}</h1>
</style>
</html>


$ jinny -t template.html --dump-to-dir "$(pwd)"
Running a build of this template /home/smashthings/jinny-tmp/template.html at: 
2023-01-23T19:42:56.581482

$ cat 0-template.html
<html>
<style>
  
  <h1> This page was generated at 2023-01-23T19:42:56.581482</h1>
</style>
</html>

basename, dirname

Straight python rips of os.path.basename and os.path.dirname


$ cat template.txt

home: {{ path.home }}
dirname: {{ path.home | dirname }}
basename: {{ (path.home + '.ssh') | basename }}

$ jinny -t template.txt
home: /home/smashthings/
dirname: /home/smashthings
basename: .ssh

removesuffix, removeprefix

These are rips of the str methods available in Python 3.9 onwards. However, you might be running a version earlier than 3.9, hence these functions are stand alone and can be used in any Python 3 version.


$ cat template.txt

removeprefix: {{ "mushroomfactory" | removeprefix("mushroom") }}
dontremoveprefix: {{ "mushroomfactory" | removeprefix("badger") }}

removesuffix: {{ "mushroomfactory" | removesuffix("ory") }}
dontremovesuffix: {{ "mushroomfactory" | removesuffix("badger") }}

$ jinny -t template.txt

removeprefix: factory
dontremoveprefix: mushroomfactory

removesuffix: mushroomfact
dontremovesuffix: mushroomfactory

censor

Censors the provided string with the ability to set the censor characters, skip censoring the first and/or last n characters or always set the censored string to x characters long. 


$ cat template.txt
---
censored_password: {{ req_envvar('PASSWORD') | censor }}
censored_password_always_10_characters: {{ req_envvar('PASSWORD') | censor(fixed_length=10) }}
censored_password_with_birds: "{{ req_envvar('PASSWORD') | censor(vals=['πŸ¦β€']) }}"
censored_password_except_first_2_characters: {{ req_envvar('PASSWORD') | censor(except_beginning=2) }}
censor_everything_except_first_and_last_characters: {{ req_envvar('PASSWORD') | censor(except_beginning=1,except_end=1) }}
passwords_are_always_five_frogs: "{{ req_envvar('PASSWORD') | censor(vals=['🐸'], fixed_length=5) }}"


$ PASSWORD=mushrooms jinny -t template.txt
---
censored_password: *********
censored_password_always_10_characters: **********
censored_password_with_birds: "πŸ¦β€πŸ¦β€πŸ¦β€πŸ¦β€πŸ¦β€πŸ¦β€πŸ¦β€πŸ¦β€πŸ¦β€"
censored_password_except_first_2_characters: mu*******
censor_everything_except_first_and_last_characters: m*******s
passwords_are_always_five_frogs: "🐸🐸🐸🐸🐸"

decorate

Through the magic of escape codes allows you to spew all sorts of horrendous colours out to terminals alongside bold, underline, blinking, strikethrough, etc. Below are the current supported codings, with background colours prefixed with `bg-` and bright / lighter colours starting with `bright`. 


$ cat colours.txt

{{ "This will look totally normal!" | decorate('normal') }}
{{ "This will be bold!" | decorate('bold') }}
{{ "This will be faint!" | decorate('faint') }}
{{ "This will be italic!" | decorate('italic') }}
{{ "This will be underline!" | decorate('underline') }}
{{ "This will be blink!" | decorate('blink') }}
{{ "This will be fastblink!" | decorate('fastblink') }}
{{ "This will be strikethrough!" | decorate('strikethrough') }}
{{ "This will be framed!" | decorate('framed') }}
{{ "This will be circled!" | decorate('circled') }}
{{ "This will be overlined!" | decorate('overlined') }}
{{ "This will be black!" | decorate('black') }}
{{ "This will be red!" | decorate('red') }}
{{ "This will be green!" | decorate('green') }}
{{ "This will be yellow!" | decorate('yellow') }}
{{ "This will be blue!" | decorate('blue') }}
{{ "This will be magenta!" | decorate('magenta') }}
{{ "This will be cyan!" | decorate('cyan') }}
{{ "This will be white!" | decorate('white') }}
{{ "This will be bg-black!" | decorate('bg-black') }}
{{ "This will be bg-red!" | decorate('bg-red') }}
{{ "This will be bg-green!" | decorate('bg-green') }}
{{ "This will be bg-yellow!" | decorate('bg-yellow') }}
{{ "This will be bg-blue!" | decorate('bg-blue') }}
{{ "This will be bg-magenta!" | decorate('bg-magenta') }}
{{ "This will be bg-cyan!" | decorate('bg-cyan') }}
{{ "This will be bg-white!" | decorate('bg-white') }}
{{ "This will be brightblack!" | decorate('brightblack') }}
{{ "This will be brightred!" | decorate('brightred') }}
{{ "This will be brightgreen!" | decorate('brightgreen') }}
{{ "This will be brightyellow!" | decorate('brightyellow') }}
{{ "This will be brightblue!" | decorate('brightblue') }}
{{ "This will be brightmagenta!" | decorate('brightmagenta') }}
{{ "This will be brightcyan!" | decorate('brightcyan') }}
{{ "This will be brightwhite!" | decorate('brightwhite') }}
{{ "This will be bg-brightblack!" | decorate('bg-brightblack') }}
{{ "This will be bg-brightred!" | decorate('bg-brightred') }}
{{ "This will be bg-brightgreen!" | decorate('bg-brightgreen') }}
{{ "This will be bg-brightyellow!" | decorate('bg-brightyellow') }}
{{ "This will be bg-brightblue!" | decorate('bg-brightblue') }}
{{ "This will be bg-brightmagenta!" | decorate('bg-brightmagenta') }}
{{ "This will be bg-brightcyan!" | decorate('bg-brightcyan') }}
{{ "This will be bg-brightwhite!" | decorate('bg-brightwhite') }}

b64encode, b64decode

Amazingly, the encode and decode functions found in Ansible are not in plain jinja. Jinny has them implemented though


$ cat template.yml
---
username: "{{ req_envvar('SECRET_USERNAME')}}"
password: "{{ req_envvar('SECRET_PASSWORD')}}"
authorization_header = "Basic {{ (req_envvar('SECRET_USERNAME') + ':' + req_envvar('SECRET_PASSWORD')) | b64encode }}"


$ SECRET_USERNAME=potus SECRET_PASSWORD=00000000 jinny -t template.yml
---
username: "potus"
password: "00000000"
authorization_header = "Basic cG90dXM6MDAwMDAwMDA="

getext

Fetches the extension for a provided path. This will take the full path, no need to filter it down with `basename` or similar. Will default to providing the leading full stop / period / '.'. This can be controlled with the `period` argument as below.


$ cat template.yml
---
getext_period: "{{ '/path/is/this.txt' | getext }}"
getext_no_period: "{{ '/path/is/this.txt' | getext(period=False) }}"


$ jinny -t template.yml
---
getext_period: ".txt"
getext_no_period: "txt"

removeext

Removes the extension from the provided path. This will take the full path and return the full path minus the extension. If you want the filename then you'll want to filter the full path to `basename` first then to `removeext`. Uses `os.path.splitext()` under the hood.


$ cat template.yml
---
removeext_short: "{{ 'this.txt' | removeext }}"
removeext_long: "{{ '/path/is/this.txt' | removeext }}"


$ jinny -t template.yml
---
removeext_short: "this"
removeext_long: "/path/is/this"

newlinetr

Replaces newlines with the provided string. Defaults to the HTML break tag `
` although other values can be provided. This is a quick function for turning multiline text into HTML or some other delimiter


$ cat template.html
---
{{ "Please make this long sentence\nWith new line characters in it\nHTML friendly!\n\n" | newlinetr }}

$ jinny -t template.html
---
Please make this long sentence<br />With new line characters in it<br />HTML friendly!<br /><br />

currency

Takes a string or a float or even an int and outputs a pretty decimal format currency string. You can do this with the baked in string format functionality within python, but this is easier to remember and allows for currency symbol changes.


$ cat template.html
---
currency_usd_str: {{ '73845400.32' | currency }}
currency_usd_float: {{ 73845400.32 | currency }}
currency_gbp: {{ '73845400.32' | currency(symbol="Β£") }}

$ jinny -t template.html
---
currency_usd_str: $73,845,400.32
currency_usd_float: $73,845,400.32
currency_gbp: "Β£73,845,400.32"

Real World Use Cases

SkySiege Security Assessments

SkySiege is a cybersecurity technology firm that provides Cloud Vulnerability Assessments and Automated Penetration Testing services that a delivered the same day. The PDF reports generated by SkySiege are fed data from SkySiege's custom scanning tools which is used as input files into HTML templates populated by Jinny. As this is all managed at a CLI level without the need for a full Jinja environment, the reports are created in a single command creating PDFs within seconds. The commands are no more complex than the examples shown in this documentation. The team at SkySiege were kind enough to provide some examples of Jinny's custom filters and globals that are used in SkySiege reports. These code snippets are provided below! You can get a sample of a SkySiege report from their sample page or you can get more details on SkySiege services at skysiege.net 


# SkySiege reports utilise SVGs for logos such as cloud provider services
# Using the file_content filter lets an SVG be loaded in a similar manner to a JSX component
<div class="absolute p-1 h-12 w-12 fill-white top-2">
  {{ (path.templatedir + '../aws/' + svc + '.html') | file_content }}
</div>

# Some reports can be hundreds of pages. SkySiege also has a number of evolving tests
# Jinny's raise_exception global ensures that reports with bad data don't ever get sent to clients
{% else %}
  {{ raise_exception("Vulnerability " + vuln.Name + " has an unknown criticality - " + vuln.Criticality) }}
{% endif %}

# As the reports are compiled to HTML the newlinetr filter is invaluable for translating breaks into strings with new lines
<p>{{ vuln.Description | newlinetr }}</p>

# For SkySiege reports to remain accessible and to reduce the blast radius on reports, SkySiege offers both censored and uncensored reports
# That way the technical team can have one report and non-technical teams can review a report without the technical details containing vulnerable infrastructure
# The censor filter is used to make censored reports possible
{% if censor is defined and (censor == true or censor == "true") %}
  <div class="truncate text-sm font-medium text-gray-800 censorResource">- {{ violation.uniqueId[-12:] | censor(except_end=3) }}</div>
{% endif %}

Globals

Globals are standalone properties that can be called like global functions or global variables. Some of Jinny's global functions will require parameters, if you miss providing these required parameters your template will helpfully and violently crash.

path

path is a global dict that is available on each template. It'll give you the variables for: - cwd - the current working directory - jinny - jinny's directory, ie the jinny module itself - template - the full path of the template currently being templated - templatedir - the directory of the template currently being templated - home - the home directory These are global values so you can access them whenever and easily combine them into paths that work locally or in unstable environments such as pipelines 


$ cat template.txt
I'm working from {{ path.cwd }}
jinny is running from {{ path.jinny }}
This template is {{ path.template }} in the directory {{ path.templatedir }}
My home directory with all my cat pics and DRG screenshots is {{ path.home }}

Rock and Stone!

$ jinny -t template.txt
I'm working from /home/smashthings/jinny-tmp/
jinny is running from /home/smashthings/.local/bin/jinny
This template is /home/smashthings/jinny-tmp/template.txt in the directory /home/smashthings/jinny-tmp/
My home directory with all my cat pics and DRG screenshots is /home/smashthings/

Rock and Stone!

time_now

time_now will generate a UTC timestamp at the point that the function is called. It will provide a microsecond ISO timestamp but with arguments you can provide a strftime compatible string to get the exact output that you're after. Python Docs 


$ cat template.txt
#### => Template Start: {{ time_now("%M:%S.%f") }}

The exact UTC time down to the microsecond is {{ time_now() }}
But if you just wanted to know what time it is for humans it's {{ time_now("%Y-%m-%dT%H:%M") }}
Or you could say {{ time_now("%A the %j day of %Y which is also the %d day of %B") }}

#### => Template Finish: {{ time_now("%M:%S.%f") }}

$ jinny -t template.txt
#### => Template Start: 28:42.871426

The exact UTC time down to the microsecond is 2022-12-31T08:28:42.871447
But if you just wanted to know what time it is for humans it's 2022-12-31T08:28
Or you could say Saturday the 365 day of 2022 which is also the 31 day of December

#### => Template Finish: 28:42.871465

prompt_envvar

prompt_envvar will prompt you for environment variables that are missing as jinny reaches them in your template(s). Once you provide a value it will set that as an environment variable and continue on, meaning that all other calls for that environment variable will receive the same value. This is useful for one off values that don't need to be committed to code or for values that you want to ask for at run time such as passwords. 


$ cat template.txt
---
I'm driving {{ prompt_envvar('destination') }} for {{ prompt_envvar('event')}}
Oh, I can't wait to see those faces
I'm driving {{ prompt_envvar('destination') }} for {{ prompt_envvar('event')}}, yeah
Well I'm moving down that line
And it's been so long
But I will be there
I sing this song
To pass the time away
Driving in my car
Driving {{ prompt_envvar('destination') }} for {{ prompt_envvar('event')}}

$ jinny -t template.txt
Please set variable 'destination':
home
Please set variable 'event':
Christmas
---
I'm driving home for Christmas
Oh, I can't wait to see those faces
I'm driving home for Christmas, yeah
Well I'm moving down that line
And it's been so long
But I will be there
I sing this song
To pass the time away
Driving in my car
Driving home for Christmas

$ destination='to the bottlo' event='a big bag of cans' jinny -t template.txt
---
I'm driving to the bottlo for a big bag of cans
Oh, I can't wait to see those faces
I'm driving to the bottlo for a big bag of cans, yeah
Well I'm moving down that line
And it's been so long
But I will be there
I sing this song
To pass the time away
Driving in my car
Driving to the bottlo for a big bag of cans

req_envvar

Checks for a required environment variable without prompting. Can take a custom message format and parameters for the printed error message if the environment variable is missing. This follows the same message format as string.format: Python Docs 


$ cat template.txt
---
super_secret: {{ req_envvar(var='PASSWORD', message_format="Need the password under var {0} yo", message_format_params=['PASSWORD']) }}

$ jinny -t template.txt
*********************
<2023-08-31T19:59:35> - TemplateHandler.Render(): Failed to render template at 'template.txt' with an exception from Jinja, details:
Type:<class 'Exception'>
Value:jinny.global_extensions.req_envvar(): Need the password under var PASSWORD yo
Trace:
...


$ PASSWORD=wizards jinny -t template.txt
---
super_secret: wizards

get_envvar

Gets an environment variable from the environment and provides a default value if not found. If a default value is not provided get_envvar will return an empty string. Returning an empty string allows for template logic to react to missing environment variables 


$ cat template.txt
---
missing_env_var: {{ get_envvar(var='BADGERS', default='NO BADGERS') }}
existing_env_var: {{ get_envvar('HOME') }}

{% if get_envvar('BADGERS') | length %}
We got {{ get_envvar('BADGERS') }} badgers from the environment variable BADGERS
{% else %}
The environment variable BADGERS didn't exist so we assume no badgers
{% endif %}

$ jinny -t template.txt
---
missing_env_var: NO BADGERS
existing_env_var: /home/smasherofallthings

The environment variable BADGERS didn't exist so we assume no badgers


$ BADGERS=40 jinny -t template.txt
---
missing_env_var: 40
existing_env_var: /home/smasherofallthings

We got 40 badgers from the environment variable BADGERS

list_files

list_files will take a directory path and will either recursively or not provide a list of all the files in that path. This is intended to be used with the `for` keyword for looping through files. Original use-case was for implementing files as keys in a Kubernetes ConfigMap. Note that the below example also makes use of path, file_content and basename, all of which are jinny additions and not in jinja. 


$ cat configmap.yml
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: loadsofscripts
data:
{% for f in list_files(path.templatedir + '/scripts') %}
  {{ f | basename }}: |
{{ f | file_content | indent(4, first=True) }}
{% endfor %}


$ cat scripts/script1.sh
#!/bin/bash
echo "This is script one"

$ cat scripts/script2.sh
#!/bin/bash
echo "This is script 2!"

$ jinny -t configmap.yml
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: loadsofscripts
data:
  script1.sh: |
    #!/bin/bash
    echo "This is script one"


  script2.sh: |
    #!/bin/bash
    echo "This is script 2!"

gen_uuid4

Generates a new version 4 UUID via the python uuid library. There's absolutely no memory on this so don't expect idempotency in your resulting templates, ie every time you run the template you'll get different UUIDs. This is awesome for generating a load of dummy data 


$ cat template.txt
{% for f in range(5) %}
{{ gen_uuid4() }}
{% endfor %}

$ jinny -t template.txt
b251f634-a912-4868-bd03-e2ccc3ae7356
4e136ef8-b106-42c9-b98c-9d61a833c523
08f9a05a-01a3-4731-a591-3e07d9876c36
7cc40da8-de29-4e37-80b9-fa3ca60ac0f0
0e044ca0-36b8-4835-ac7c-c7fa5215fdcf

b64file

Reads the binary content of a file and returns the base64 encoded string. This was originally built for images stored in their raw format but required loaded directly into HTML, allowing a single function to embed an image directly into a document. Yes, that is a base64 encoding of a garbage fire. 


$ cat garbage-fire.html
<h1>Look, a garbage fire!</h1>
<img src="data:image/png;base64,{{ b64file(path.templatedir + '/img/garbage-fire.png') }}" />

$ jinny -t garbage-fire.html
<h1>Look, a garbage fire!</h1>
<img src="data:image/png;base64,/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/wAARCACAAIADASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwDFPTFN2mjafSnYOzFdj16Hsr92rKS1YhU4A4o2H2/Ok2n0qTaSBjr9aTKUnGyUlqOQhVwadke9IIDnJPFSGNT0H61LktyeSzS5kVjGxOcUbCDnirHlD0/WkaIHkEihST1K1S5OZbEOKMc0/wAorycYqNkIq+byMfZq/LzITafb86ULweRzSKCM8dqTafSla2tjVy5m4uSF2H2/OlAwMGgA7TTdp9Ke2tgk+dOLkhJYi+CpAYeveoWEkY3Oo29ypzirG07aQqSMYyKrnt0Of2EZauQvO09aAD6Gk3N61IoJA5qNGdEnKGumrHImRzxU6xngdKSKMkg44FWAgBPb8KwqySW46U5X5bIEi3naG2mhoXU4ZiB6+tSxcyrj3rVWyE1qWAyw6j1ryquMlTne+h6MKEfZ2luYu3pgjFMKdasPEySbMZPam4xwRz3reFeMpXTMpUpxhy2XqVXRs5zkUwqSMGrTJgHpj6VGwOOPyxXdGstGcMqE37t0UipB6Ggd6sFc+g/CoHDKe9dCmrGfsJt20I/m96cM7TQCSDmimojnVlflstGNw3oacPuj1oJIXrTcn1NGkWU+atHotRwJKmp4x8gPGagVj7VbQYANZTlpuaRhyvWK1/rsTIOnHNSYAIwO9MU9c+lWIoHLKT0615mIqpLc7KFJp35UWYrYyuXUcgc4roNIsZJrdiOOelVNKtXE5DKQMfnXbadZiFkKL8p5zXz+IqtvkRWLxHIrI4u80thcMSMDH61h3FuwfGOQcH2r1K808Sl2C4AGc1zF7o4WR3K5DDjFFLEyi7SFQxcZq0jj5IcZK8+1Vp2jij3uwUZAyas6pcfY5TDGA0x7dlHqa59o1llwVEpBy7t/nrX0+X06lSHNLY4Mxr0YS5YLU1PMjlj3wlXUHBqrPPHHw7gE8gVS8qJpCscShAfmb+gpDHE0nyxrtX77nt7V6caNnuedLFpxty6lnfkAqcg96duO0nPNSWthbRQi/vI9sC/6uHHMh+np6Cq8bHYMAAHkD09qUo8u7NcPi44qbjCHw21Hbj604H5QaTPy9BSbjUppbs6pwc1aMUrP+ug9T8pOOlWVboaqB8cYqzG2V6VnPVaM0jFJ6x66f1ctpjI9zW9aWqzRE5wc4FcxJMUjLZK4IyQM4GeSB9K0oLm4hGYrq8APP/HqD/7LXm4jA1cTrTaJrZpRwS5Kid3rtc9F0uwyinGT6+lb8MflRhc815Zb+JNXtgVjvrrHobJf/ian/wCEt1v/AJ/rn/wBH/xNcKyHEp3uvv8A+AeLVzijN31+49PZQylT0Ncf4z8Q22lW4s7bbJfsOF6iNf7zf0HeufPi3WyMfbrkf9uI/wDia5uWy86aSaW61B5ZGLO7W+SxPc/LXVhskaneva3ZdTGWa0kvdv8AcZskkk8rjezMTmSU9Sf8f5U3748qH5Y1OGYd/Yf41pDTo/8Anvff+A//ANjSf2dH/wA977/wH/8Asa+jUUlZHM8fSbu2ZbkEGOM7Y04Zx/IVfs7OOOJbq6QiAY8iDHMh7Ej+n51KLS0tSsk32uYA/LG8O0M3bPApTI8twHm5kZTgA8IOOB+fWqUbnFjMwXLaA2d2nZ3nOZSh2qOiD0H+NUojmJTjsKtyf65v+uRqvCg8lP8AdH8qitZWOrh9uSqPq+X9Qz9KQnAzTmAXoM1H5qk7cgn0zWDasfSwg7p269/+CVRe8j91nPORnH8qeuoMG2LHlsZwM/4VHGCY0YSAgqODIQc96SQIofe6l9pwQ3ftWvLHaxzc9TfmL8k6vZs2OWQ4AyecV06+YYI/LKg7R94Z7VxtteLbOzmJ2QxPHgkHbnuP8967SD/j3i/3B/KnCHKeBnNeVSUU1ohmLn+/D/3wf8aXFz/fh/74P+NQas1yulXBtAfPC/Ljr74/CsXwnNqMrXAumlaEAbTLn73tmrPKjS5qbnpodDi5/vw/98H/ABpMXP8Aeh/74P8AjVV9bsYppop5vJeI4IkGM+49ac93Jd6RLcWKuJGRvKDLgk9uKCfZy0urFjFz/fh/74P+NGLn+/D/AN8H/Gua8KzanJezLcmZoNvJlzw2e2fxrrKB1oOlPl0Zm6pvFrD5m0t5w+6MDvWW83+kjYjyFVIOxc4ORxWvq/8Ax6x+0q1lQXEtoDCIS7HJDDHIz15PvVx2MZJNpsPJvJnLrAEBXb87f4f41VBeE+TJjcijp0I6Z/SrT3d47lAqIdu75mz/AC/xrIR5r2TID+ay8kMAqj/CoqK8T18lnOFX3fh6/oWgryzFVTeAnC5x3+oqqYpWEa/MQnT39O/8qtWcgivHBdkYphDIdw3e+O1AyduzaD8mCc4P4fTNYwunqe/VmpSvEobV/uj8qAqjoo/KlorqOQa/3G+ldxFPHHawb2xuQEce1cQwypHqK24vE8cMMURtWyoCk7xj09Klnn46lOoo8qub32yD+/8A+Omj7ZB/f/Q1lL4h3oHWzJBGR+8H+FRP4qjQgfZS2VDfLIDjPY8dak876pV25Xr5mrK9jMwaVI3ZehaPOP0qUXduBjf/AOOmsMeLIz/y5v8A99ij/hLI/wDnzf8A77FMr6nW/lf3m59sg/v/APjpo+2Qf3//AB01ip4qR22izYH3kHqB/Wpj4gK4zaEZIGfM9fwpEvC1Vf3S3qjB7KNlOQZUx+dZxH+kof8AZb+YqtqPiATIYBakMkgOd/BwfpVH+2JXJyojHbau79f/AK1WnZEvAV6lnFGsynz92Pl2EZrItHjFrIjybHIUqc4yB2qtJeTSud0spUZ/i25/AUIysMAdOxpSXMj1cuwtTDXc+o2R1w37zcx44NWUuyqriEkqB1b0H0qLA9BQWCjk4FFjvuLRRRVCCmmNW6jmnUUAR+Vj7ruvsGNNEGB96pjnHHWmgvnkDHsaVguyPyT/AHhR5J9RU1FFguQmAkdRTisrKVaZip6gnrUlNLqvWiwXIxDgYyPwFL5P+1+lPDMSMIfx4pwSY8+UcetGg9SMRL7mnhQo4FJlgcFPwByfyoDgnBBBoEOooppdR1YUwHKCzqoxk9ycAUj5jJDjgHG4HINBUH6+tNEajk/MfU0tbj0sPooopiCiiigAooooAKaHeMj5cgZwR7+tOopAJ54yHPDDj5if8KTzYxwST77v/rU6iiw7iedGAoUDA544JP5UGZnIwhODkZyBS03D8ciiwriqCFxnmkTzYtvlyYK9MqDij5/UUoDcZI96LJ7jTa2P/9k=" />

is_file

Returns true if the path provided both exists and resolves to a file. Good for condition inclusions or templating. 


$ ls .
index.html        optional.css

$ cat index.html
{% if is_file(path.templatedir + '/optional.css') %}
<style>
/* optional.css */
{{ (path.templatedir + '/optional.css') | file_content }}
</style>
{% endif %}

{% if is_file(path.templatedir + '/no-exist.css') %}
<style>
/* no-exist.css */
{{ (path.templatedir + '/no-exist.css') | file_content }}
</style>
{% endif %}

$ cat optional.css
.title {
  color: red;
}

$ jinny -t index.html
<style>
/* optional.css */
.title {
  color: red;
}
</style>

is_dir

Returns true if the path provided both exists and resolves to a directory. Not sure what you'll use this for but go nuts 


$ ls .
index.html        directory

$ cat index.html
{% if is_dir(path.templatedir + '/directory') %}
<h2>
A directory exists at {{ path.templatedir + '/directory' }}
</h2>
{% endif %}

$ jinny -t index.html
<h2>
A directory exists at /home/smashthings/jinny-tmp/
</h2>

raise_exception / throw

Raises an Exception. Amazingly not included within Jinja without some ugly hacks so here's a nicer function that allows for clean and clear exceptions 


$ cat template.txt
{{ raise_exception() }}

$ jinny -t template.txt
*********************
<2024-07-19T14:18:48> - TemplateHandler.Render(): Failed to render template at '/home/smashthings/template.txt' with an exception from Jinja, details:
Type:<class 'Exception'>
Value:Template raised an exception using the 'raise_exception' function!
Trace:
Traceback (most recent call last):
  File "/home/smashthings/jinny.py", line 488, in Render
    self.result = self.loadedTemplate.render(values)
  File "/home/smashthings/.local/lib/python/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/home/smashthings/.local/lib/python/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
  File "/home/smashthings/jinny/imports/global_extensions.py", line 69, in raise_exception
    raise Exception(exc)
Exception: Template raised an exception using the 'raise_exception' function!


$ cat mortgage_application_response.txt
{{ raise_exception("computer says no") }}

$ jinny -t mortgage_application_response.txt
*********************
<2024-07-19T14:19:29> - TemplateHandler.Render(): Failed to render template at '/home/smashthings/template.txt' with an exception from Jinja, details:
Type:<class 'Exception'>
Value:computer says no
Trace:
Traceback (most recent call last):
  File "/home/smashthings/jinny.py", line 488, in Render
    self.result = self.loadedTemplate.render(values)
  File "/home/smashthings/.local/lib/python/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/home/smashthings/.local/lib/python/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
  File "/home/smashthings/jinny/imports/global_extensions.py", line 69, in raise_exception
    raise Exception(exc)
Exception: computer says no

Frontend

One of the unexpected use cases for Jinny is in creating static sites, particularly when completely compiling all aspects to a single HTML file including JS, CSS, images and even fonts. Frontend frameworks can be heavy and often bring the full baggage of the Javascript ecosystem for what is a simple templating task.

Given that most frontend directory structures break out images, fonts, code and other components into separate directories with clear import processes it's simple to put together various Jinny filters and global functions to build that data directly into a single page without needing multiple artifacts to be deployed nor a chunky framework to update.

A great pattern is to have Jinny perform the basic templating in development allowing for development use of frameworks such as the TailwindCSS CDN without having a full rebuild and treeshake for each change.

Another aspect is that by compiling all or selected content into a single page frontend, development work can proceed without having to load in a full backend. Jinny can compile everything the frontend needs such as styling, images and more into a single file loaded directly by the browser without needing a local server.

All of this depends on your application. Most applications from the 2020s with complex frontends will be using JSX Components. Jinny won't replace that, but what it can do is add independent tools allowing for easy translation of frontend artefacts into your resulting HTML.

Here's some examples:

Development Flag

Adding a conditional that looks for a flag to indicate development can be used for pulling in extended development code, metadata and more for local development. This is used on this very site for pulling in the TailwindCSS CDN when run in development mode and not requiring a full scan and compilation of TailwindCSS. When run this way all TailwindCSS classes are available preloaded in the development browser allowing for manual testing via edits without a full rebuild. Also included is setting a Javascript variable via script to indicate that debug mode is activated.


{% if dev_env is defined %}
<script src="https://cdn.tailwindcss.com"></script>
<script>window.project.debug = true;</script>
{% else%}

Component Code Snippets

Likewise, you may wish to only load Google Analytics in development. Using a development flag works the same way here but with the added logic that if a Google Analytics code is provided in the inputs under `config.ga_code` then Google Analytics will be loaded. By simply providing your GA Code this generates the full implementation.


{% if dev_env is not defined and config.ga_code %}
<!-- Google Analytics via GTag -->
<script async src="https://www.googletagmanager.com/gtag/js?id={{ config.ga_code }}"></script>
<script>
  window.dataLayer = window.dataLayer || [];
  function gtag(){dataLayer.push(arguments);}
  gtag('js', new Date());

  gtag('config', '{{ config.ga_code }}');
</script>
{% endif %}

Mass Importing JS and CSS

Any file can be imported into a template via the `file_content` filter, however, when combined with the `list_files` function you can dynamically import CSS and JS files into your HTML directly. This format below leaves the HTML tags in the template, hence each imported .css and .js file can be pure CSS / JS.


<!-- Importing all the CSS files in ./css/ -->
{% for f in list_files(path.templatedir + '/css/') %}
<style>
  {{ f | file_content }}
</style>
{% endfor %}

<!-- Importing all the JS files in ./js/ -->
{% for f in list_files(path.templatedir + '/js/') %}
<script>
  {{ f | file_content }}
</script>
{% endfor %}

Adding Fonts

Fonts require import in a particular manner as the `@font-face` styling requires naming and determination of it's type. Using a few filters this can be generated automatically for either one file or for a list of files in a directory by reading the extension and file's basename of the font to determine the name and type of the font and by loading the base64 encoded version of the font.


<!-- Importing a single font from the ./fonts/NewFont.ttf as a TrueType Font with name NewFont -->
<style>
@font-face {
  font-family: {{ path.templatedir + '/fonts/NewFont.ttf' | basename | removeext }};
  src: url(data:font/{{ path.templatedir + '/fonts/NewFont.ttf' | getext(period=False) | lower }}; base64, {{ b64file(path.templatedir + '/fonts/NewFont.ttf') }});
}
</style>

<!-- Importing all the font files in the ./fonts/ directory with each filename becoming the font name -->
<style>
{% for f in list_files(path.templatedir + '/fonts/') %}
@font-face {
  font-family: {{ f | basename | removeext }};
  src: url(data:font/{{ path.templatedir + '/fonts/' + f | getext(period=False) | lower }}; base64, {{ b64file(f) }});
}
{% endfor %}
</style>

Embedding Images

Images can be embedded simply via their base64 content. This allows for images to be built directly into the page without needing separate calls for each individual image. Obviously, you'll lose lasy loading with this approach, but for sites that are light on images or for selected small images this is a great approach to simply loading in the image without also having to deploy each image file.


<!-- Loading in the favicon directly from file './logo.png' -->
<link as="image" href="data:image/png;base64,{{ b64file(path.templatedir + '/logo.png') }}">

<!-- Embedding a full image from picture.jpg -->
<img src="data:image/{{ path.templatedir + '/picture.jpg' | getext }};base64,{{ b64file(path.templatedir + '/picture.jpg') }}">

SVG Components

SVGs are common as icons and are essentially big bundles of HTML. Because they're just HTML we can import them as per any other file but also template them if they have variables such as common width or height values


<!-- Importing an SVG as is from './logo.svg' -->
{{ path.templatedir + '/logo.svg' | file_content }}

<!-- Importing an SVG at './logo.svg' that contains internal variables as a nested template -->
{{ path.templatedir + '/logo.svg' | nested_template }}

Unsafe

Unsafe is a set of filters / globals / functions that can only be operated if the -u or --unsafe CLI parameter is provided.

Do not use unsafe functions unless you know exactly what they're doing.

No, really. They have a nigh unthinkable blast radius.

Unsafe functions are not executed unless you provide the unsafe CLI argument. The packages that contribute to unsafe functions - eg subprocess - are not imported unless the CLI argument is provided. If you try to run a template that makes use of unsafe functions without providing the CLI you'll get a full exception and Jinny will crash out.

*********************
<2024-07-14T20:56:33> - TemplateHandler.Render(): Failed to render template at '/home/smashthings/template.txt' with an exception from Jinja, details:
Type:<class 'Exception'>
Value:jinny_unsafe.Cmd(): "cmd" is an unsafe function and thus requires the -u / --unsafe CLI argument to utilise. This is done to protect you from potentially malicious execution
Trace:
Traceback (most recent call last):
  File "/home/smashthings/jinny/jinny.py", line 485, in Render
    self.result = self.loadedTemplate.render(values)
  File "/home/smashthings/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/home/smashthings/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "<template>", line 1, in top-level template code
  File "/home/smashthings/jinny/imports/jinny_unsafe.py", line 7, in Cmd
    raise Exception(f'jinny_unsafe.Cmd(): "cmd" is an unsafe function and thus requires the -u / --unsafe CLI argument to utilise. This is done to protect you from potentially malicious execution')
Exception: jinny_unsafe.Cmd(): "cmd" is an unsafe function and thus requires the -u / --unsafe CLI argument to utilise. This is done to protect you from potentially malicious execution

cmd

cmd takes either a single string or a list to pass through to subprocess.run. This is done as a way to dynamically pull in content without Jinny needing to include every conceivable piece of code in existence. Original use case was to print out the current git hash for repo documentation which is demonstrated below. It's in unsafe for really obvious reasons. Read the documentation for the subprocess module to get an idea of how you can get utterly wrecked by calling random commands such as the permissions that are granted, disk access, network access, etc. Only allow use of this function in your templates if you trust the templates. In addition to a string or list making up a command cmd also allows for flexibility in the output so you can control whether the command is loaded into a shell environment, just getting stdout, stderr or the exit status and whether or not you want the command to crash out if the command fails. Function params are: - shell:bool=True -> chose to run the command in a shell environment, check out the subprocess module for more details - quitOnFailure:bool=True -> raises an exception if the command fails - returnStatusCode:bool=False -> whether to output the resulting status code - returnStdOut:bool=True -> whether to output stdout - returnStdErr:bool=True -> whether to output stderr - outputJoin:str="\n" -> for each of statuscode, stdout and stderr what should be the separator when optionally printing them in that order - raw:bool=False -> Raw forces raw output as per the below notes An example showing the full output potential is below. Also note that: The raw argument overrides some of the formatting safety that is included by default, namely: - If stderr is empty then stderr will not be printed at all. As shown in the example below where returnStdErr is requested with a | separator but only the status code and stdout are included. This is done to avoid introducing impossible to debug spacing issues - StdOut and StdErr from the command are .strip() of white space characters at the start and the end of each. Once again this is done to avoid debugging whitespace from commands. Jinny is a templating tool so whitespace is best managed in Jinny and not by commands 


$ cat template.txt
The git hash of this repo is {{ cmd('git rev-parse HEAD') }}
A full raw output including newlines from the command and stderr even if it's empty is {{ cmd('git rev-parse HEAD', returnStatusCode=True, returnStdOut=True, returnStdErr=True, outputJoin='|', raw=True) }}
A full output without whitespace nonsense would be {{ cmd('git rev-parse HEAD', returnStatusCode=True, returnStdOut=True, returnStdErr=True, outputJoin='|') }}

$ jinny -t template.txt --unsafe
The git hash of this repo is 76bc7a3dad0616f8ee9f2d8d76a007454d49c531
A full raw output including newlines from the command and stderr even if it's empty is 0|76bc7a3dad0616f8ee9f2d8d76a007454d49c531
|
A full output without whitespace nonsense would be 0|76bc7a3dad0616f8ee9f2d8d76a007454d49c531

Don't do anything stupid.

Errata

Input Files

Jinny will read the file extension of provided input files to work out if they're JSON, YAML or .env files. If you don't have an extension on the filename then jinny will attempt parsing in YAML and then JSON. It won't try .env as it's a very easy to misinterpret .env files. Environment files MUST end with `.env` to be parsed as environment files. Jinny has a very simple logic to parsing environment files as inputs into jinny - Jinny will read the file line by line - Each line will be split at the first `=`. Therefore your values can contain `=`. This is a dumb split and thus won't care if your keys are non-POSIX. See below for examples - Jinny will make a fair effort to handle multi-line values. Read below for more details - Jinny will skip commented lines - ie. starting with a `#` - Jinny will ignore blank lines

Multi-line Values

Jinny makes a fair effort to interpret multi-line values. Ideally you *shouldn't be doing this* as multi-line values get interpreted differently across a whole host of shells and other programmes. You *should* be base64 encoding your values that have special characters - including new line characters - and setting the encoded result as the value. However, I have seen multi-line values used so often that Jinny will make a fair effort to handle them. There's no promises that this will do exactly what you want and you need to be wary of new lines added / interpreted / missed at the end of your value.

Packages used

Check out src/jinny/requirements.txt As of December 2022 only PyYAML and Jinja2 are used outside the standard library

FAQs

Will jinny ever integrate with Kubernetes directly? No. See below for an example of a workable approach. What about Windows? My Windows days are behind me and I'm not coming back to it. If you'll like to PR it in go for it, however I'm neither motivated nor tooled up to maintain support for Windows. Can I donate or support? Nah. I'm good. If jinny really helps and you want to right the balance then please donate to the Guide Dogs UK or Australia or Tassie. With some time, energy and some delicious dog treats they give people back their independence, it's the best ROI I know of. Guide Dogs UK Guide Dogs AU Guide Dogs Tasmania Can you offer support? jinny is simple enough that your problem isn't likely to be jinny itself. If it is then open up an issue here on Gitlab or on Github.

Why?

The 2020's of software usually include mashing together different/underlying/proxied systems that need to be able to scale, adapt and transform in unstable environments (no pets, black box providers, etc) and unstable direction. This means you're running applications and controlling services that lead to a mass of config that needs to change on a whim. Add to this the need to pass this through various CI/CD pipelines and there's a need for a templating application that is: - Command line controlled - Can take multiple JSON & YAML template inputs - Can take multiple Jinja based templates - Can choose to template out to stdout, to separate files, to one big file - Allows for cascading overwriting of inputs - Allows the utilising of a seasoned templating language with some room for adding functionality - Stable - uses simple and reliable libraries and doesn't need constant maintenance. We don't want this failing our pipelines or botching deployments For example, Jinny was originally conceived as a way to handle templating of Kubernetes manifests rather than using Helm or other Go templating tools. Helm is over-engineered for what I often need and usually comes with unwanted issues such as nuking production environments (your mileage may vary). Jinny doesn't attempt Kubernetes package management, whatever that is, and instead just sticks to templating such that you as the Ops engineer can choose how, when or what to apply.

Kubernetes

With the move to Kubernetes the amount of templating and general boilerplate become quite heavy going. There's less coding of systems and more grabbing what's on the OSS shelf and slamming config into it until it does what you need it to do. I understand the reasoning for it, but a major side effect is that what there's less 'writing code' and more 'managing config'. The dominant technology for managing Kubernetes configuration is Helm. Helm and Jinny both do templating. Helm will also attempt deployment management, but I've also found it used just for the templating. A large motivator behind Jinny was the state of Helm. During my professional career I've witnessed Helm commit significant financial damage through its unreliability, poor logic and its flawed attempts to do too much. Helm templating is a significant downgrade to what Jinja can provide. Helm's deployment management is a source of woe. You can read more about this [here.](https://southall.solutions/articles/helm-an-overview/) Additionally, templating related requirements have arisen for which there hasn't been suitable tooling. Email templating. Generating NGINX configurations based on the existence of relevant environment variables. Creating reports based on environment variables that can censor sensitive data as required. Jinny has grown to cover these additional use cases as well as handle Kubernetes. Jinny doesn't interface directly with Kubernetes as it only handles the templating. How you deploy to your clusters is instead left entirely up to you. Therefore the basic, lowest investment interaction is essentially: jinny template * | kubectl -f - You can do this with a couple of shell functions with the caveat that your templates cannot take inputs or your function must include standardised named input files. Given that Jinny templates can be executed without any input files at all this is still a viable option. jk is the name I gave the function but you can use whatever you want. Add the below to your shell's run command script at ${HOME}/.bashrc or similar. The stdout-separator argument in the below functions places YAML separators on each file that you pass through, meaning that you can do cool things like mash in various files and have them all apply at once. The caveats with this approach being: - There's no input files - Relies on tempfile and kubectl being installed - Writes a file to disk or wherever volume tempfile is configured to write to - It's compatible with my bash/zsh setup but you need to check your own I'm cool with all of that so works well for me. 


function jk(){
  tmp=$(tempfile)
  jinny --stdout-seperator='---' -t "${@}" > $tmp
  if [[ $? == "0" ]]; then
    kubectl apply -f $tmp
  else
    cat $tmp
  fi
  rm -rf $tmp
}

function jd(){
  tmp=$(tempfile)
  jinny --stdout-seperator='---' -t "${@}" > $tmp
  if [[ $? == "0" ]]; then
    kubectl delete -f $tmp
  else
    cat $tmp
  fi
  rm -rf $tmp
}