2  Why use Julia?

Alec Loudenback

We want a language that’s open source, with a liberal license. We want the speed of C with the dynamism of Ruby. We want a language that’s homoiconic, with true macros like Lisp, but with obvious, familiar mathematical notation like Matlab. We want something as usable for general programming as Python, as easy for statistics as R, as natural for string processing as Perl, as powerful for linear algebra as Matlab, as good at gluing programs together as the shell. Something that is dirt simple to learn, yet keeps the most serious hackers happy. We want it interactive and we want it compiled. - Bezanson, Karpinski, Shaw, Edelman (the original creators of Julia)

2.1 Chapter Overview

We motivate the use of Julia as a preferred language for financial modeling.

2.2 Introduction

Julia is a relatively new¹, productive, and fast programming language. It is evident in its pragmatic, productivity-focused design choices, pleasant syntax, rich ecosystem, thriving communities, and its ability to be both very general purpose and power cutting edge computing.

With Julia: math-heavy code looks like math; it’s easy to pick up, and quick to prototype. Packages are well-integrated, with excellent visualization libraries and pragmatic design choices.

Julia’s popularity continues to grow across many fields and there’s a growing body of online references and tutorials, videos, and print media to learn from.

Large financial services organizations have already started realizing gains: BlackRock’s Aladdin portfolio modeling, the Federal Reserve’s economic simulations, and Aviva’s Solvency II-compliant modeling². The last of these has a great talk on YouTube by Aviva’s Tim Thornham, which showcases an on-the-ground view of what difference the right choice of technology and programming language can make. Moving from their vendor-supplied modeling solution was 1000x faster, took 1/10 the amount of code, and was implemented 10x faster.

The language is not just great for data science — but also modeling, ETL, visualizations, package control/version management, machine learning, string manipulation, web-backends, and many other use cases.

Julia is well suited for financial modeling work: easy to read and write and very performant.

Tip

The two language problem is a term describing processes and teams that separate “domain expertise” coding from “production” coding. This isn’t always a “problem”, but the “two language problem” describes the scenario where this arises not out of intention but out of the necessity of dealing with limitations of the programming languages used. The most common combination is when the domain experts utilize Python, while the quants or developers write C++. This arises because the productive, high level language hits a barrier in terms of speed, efficiency, and robustness. Then, as a necessary step to achieve the end goals of the business, the domain experts hand off the logic to be re-implemented into the lower level language. Not only does this effectively limit the architecture, it essentially defines a required staffing model which may introduce a lot of cost and redundancy in expertise. Julia solves this to a large extent, allowing for a high level, productive language to be very fast.

A similar, related dichotomy is the two culture problem wherein domain experts (e.g. financial analysts) exist in a different sphere from developers. This manifests in many ways, such as restricting the tools that each group is permitted to use (e.g. Excel for domain experts, codebases and Git for developers). This is less of a technical problem and more of a social one. However, Julia is also one of the better languages in this regard, because much of the associated tooling is made as straightforward as possible (e.g. packaging, distribution, workflows, etc.). See Chapter 12 and Chapter 21 through Chapter 24 for more on this.

2.3 Julia and This Book

Julia is introduced insofar that a basic understanding is necessary to illustrate certain concepts. Julia is ideal in this context, because it is generally straightforward and concise, allowing the presented idea to have the spotlight (as opposed to language boiler plate or obtuse keywords and variables). The point of structuring the book like this is to allow us to introduce a wide variety of computer science concepts to the financial professional, not to introduce Julia as a programming language (there are many other resources which do that just fine).

This chapter seeks to motivate to the skeptical professional why we choose Julia for teaching and for work. Then, in Chapter 5 we provide an introduction to core language concepts and syntax. After this chapter,the content is focused on illustrating a number of key concepts, with Julia taking a secondary role, serving simply as a backdrop. It’s not until Chapter 21 where Julia regains the spotlight and we discuss particulars which generally matter only to those heavily using Julia more vigorously.

2.4 Expressiveness and Syntax

Expressiveness is the manner in which and scope of ideas and concepts that can be represented in a programming language. Syntax refers to how the code looks on the screen and its readability.

In a language with high expressiveness and pleasant syntax, you:

• Go from idea in your head to final product faster.
• Encapsulate concepts naturally and write concise functions.
• Compose functions and data naturally.
• Focus on the end-goal instead of fighting the tools.

Expressiveness can be hard to explain, but perhaps two short examples will illustrate.

2.4.1 Example 1: Retention Analysis

This is a really simple example relating Cessions, Policys, and Lives to do simple retention analysis. Retention is a measure of how much risk an insurance company holds on a policy after it’s own reinsurance risk transfer (ceded amount of coverage are called “cessions”).

First, let’s define our data:

# Define our data structures
struct Life
    policies
end

struct Policy
    face
    cessions
end

struct Cession
    ceded
end

Now to calculate amounts retained. First, let’s say what retention means for a Policy:

# define retention
function retained(pol::Policy)
    pol.face - sum(cession.ceded for cession in pol.cessions)
end

retained (generic function with 1 method)

And then what retention means for a Life:

function retained(l::Life)
    sum(retained(policy) for policy in life.policies)
end

retained (generic function with 2 methods)

It’s almost exactly how you’d specify it English. No joins, no boilerplate, no fiddling with complicated syntax. You can express ideas and concepts the way that you think of them, not, for example, as a series of dataframe joins or as row/column coordinates on a spreadsheet.

We defined retained and adapted it to mean related, but different things depending on the specific context. That is, we didn’t have to define retained_life(...) and retained_pol(...) because Julia can dispatch based on what you give it (this is a more powerful, generalized version of method dispatch commonly used in object-oriented programming, see Chapter 7 for more).

Let’s use the above code in practice then.

# create two policies with two and one cessions respectively
pol_1 = Policy(1000, [Cession(100), Cession(500)])
pol_2 = Policy(2500, [Cession(1000)])

# create a life, which has the two policies
life = Life([pol_1, pol_2])

Life(Policy[Policy(1000, Cession[Cession(100), Cession(500)]), Policy(2500, Cession[Cession(1000)])])

retained(pol_1)

400

retained(life)

1900

And for the last trick, something called “broadcasting”, which automatically vectorizes any function you write, no need to write loops or create if statements to handle a single vs repeated case:

retained.(life.policies) # retained amount for each policy

2-element Vector{Int64}:
  400
 1500

2.4.2 Example 2: Random Sampling

As another motivating example showcasing multiple dispatch, here’s random sampling in Julia, R, and Python.

We generate 100:

• Uniform random numbers
• Standard normal random numbers
• Bernoulli random number
• Random samples with a given set

Table 2.1: A comparison of random outcome generation in Julia, R, and Python.

Julia	R	Python
using Distributions rand(100) rand(Normal(), 100) rand(Bernoulli(0.5), 100) rand(["Preferred","Standard"], 100)	runif(100) rnorm(100) rbern(100, 0.5) sample(c("Preferred","Standard"), 100, replace=TRUE)	import scipy.stats as sps import numpy as np sps.uniform.rvs(size=100) sps.norm.rvs(size=100) sps.bernoulli.rvs(p=0.5,size=100) np.random.choice(["Preferred","Standard"], size=100)

By understanding the different types of things passed to rand(), it maintains the same syntax across a variety of different scenarios. We could define rand(Cession) and have it generate a random Cession like we used above.

2.5 The Speed

As stated in the journal Nature, “Come for the Syntax, Stay for the Speed”.

Earlier we described Aviva’s Solvency II compliance modeling, which ran 1000x faster than the prior vendor solution mentioned earlier: what does it mean to be 1000x faster at something? It’s the difference between something taking 10 seconds instead of 3 hours — or 1 hour instead of 42 days.

With this difference in speed, you would be able to complete existing processes much faster, or extend the analysis further. This speed could allow you to do new things: a stochastic analysis of life-level claims, machine learning with your experience data, or perform much more frequent valuation.

Here’s a real example, comparing the runtime to calculate the price of a vanilla European call option using the Black-Scholes-Merton formula, as well as the associated code for each. Here’s the mathematical formula we are using:

\[ \begin{aligned} \text{Call}(S_t, t) &= N(d_1)S_t - N(d_2)Ke^{-r(T - t)} \\ d_1 &= \frac{1}{\sigma\sqrt{T - t}}\left[\ln\left(\frac{S_t}{K}\right) + \left(r + \frac{\sigma^2}{2}\right)(T - t)\right] \\ d_2 &= d_1 - \sigma\sqrt{T - t} \end{aligned} \]

using Distributions

function d1(S,K,τ,r,σ)
    (log(S/K) + (r + σ^2/2) * τ) / (σ * √(τ))
end

function d2(S,K,τ,r,σ)
    d1(S,K,τ,r,σ) - σ * √(τ)
end

function Call(S,K,τ,r,σ)
    N(x) = cdf(Normal(),x)
    d₁ = d1(S,K,τ,r,σ)
    d₂ = d2(S,K,τ,r,σ)
    return N(d₁)*S - N(d₂) * K * exp(-r*τ)
end

S,K,τ,σ,r = 300, 250, 1, 0.15, 0.03

Call(S,K,τ,r,σ) # 58.81976813699322

from scipy import stats
import math

def d1(S,K,τ,r,σ):
    return (math.log(S/K) + (r + σ**2/2) * τ) / (σ * math.sqrt(τ))

def d2(S,K,τ,r,σ):
    return d1(S,K,τ,r,σ) - σ * math.sqrt(τ)

def Call(S,K,τ,r,σ):
    N = lambda x: stats.norm().cdf(x)
    d_1 = d1(S,K,τ,r,σ)
    d_2 = d2(S,K,τ,r,σ)
    return N(d_1)*S - N(d_2) * K * math.exp(-r*τ)

S = 300
K = 250
τ = 1
σ = 0.15
r = 0.03

Call(S,K,τ,r,σ) # 58.81976813699322

d1<- function(S,K,t,r,sig) {
  ans <- (log(S/K) + (r + sig^2/2)*t) / (sig*sqrt(t))
  return(ans)
} 

d2 <- function(S,K,t,r,sig) {
  return(d1(S,K,t,r,sig) - sig*sqrt(t))
}

Call <- function(S,K,t,r,sig) {
  d_1 <- d1(S,K,t,r,sig)
  d_2 <- d2(S,K,t,r,sig)
  return(S*pnorm(d_1) - K*exp(-r*t)*pnorm(d_2))
}
S <- 300
K <- 250
t <- 1
r <- 0.03
sig <- 0.15

Call(S,K,t,r,sig) # 58.81977

We find in Table 2.2 that despite the syntactic similarity, Julia is much faster than the other two.

Table 2.2: Julia is nearly 20,000 times faster than Python, and two orders of magnitude faster than R.

Language	Median (nanoseconds)	Mean (nanoseconds)	Relative Mean
Python	not calculated by benchmarking library	817000.0	19926.0
R	3649.0	3855.2	92.7
Julia	41.0	41.6	1.0

2.5.1 Development Speed

Speed is not just great for improvement in production processes. During development, it’s really helpful too. When building something, the faster feedback loop allows for more productive development. The build, test, fix, iteration cycle goes faster this way.

Admittedly, most workflows don’t see a 1000x speedup, but 10x to 1000x is a very common range of speed differences vs R or Python or MATLAB.

Note

Sometimes you will see less of a speed difference; R and Python have already circumvented this and written much core code in low-level languages. This is an example of what’s called the “two-language” problem where the language productive to write in isn’t very fast. For example, more than half of R packages use C/C++/Fortran and core packages in Python like Pandas, PyTorch, NumPy, SciPy, etc. do this too.

Within the bounds of the optimized R/Python libraries, you can leverage this work. Extending it can be difficult: what if you have a custom retention management system running on millions of policies every night? In technical terms, libraries like NumPy are not able to handle custom data types, and instead limit use to pre-built types within the library. In contrast, all types in Julia are effectively equal, even ones that you might create yourself.

Julia packages you are using are almost always written in pure Julia: you can see what’s going on, learn from them, or even contribute a package of your own!

2.6 More of Julia’s benefits

Julia is easy to write, learn, and be productive in:

• It’s free and open-source
  • Very permissive licenses, facilitating the use in commercial environments (same with most packages)
• Large and growing set of available packages
• Write how you like because it’s multi-paradigm: vector-izable (R), object-oriented (Python), functional (Lisp), or detail-oriented (C)
• Built-in package manager, documentation, and testing-library
• Jupyter Notebook support (it’s in the name! Julia-Python-R)
• Many small, nice things that add up:
  • Unicode characters like α or β
  • Nice display of arrays
  • Simple anonymous function syntax
  • Wide range of text editor support
  • First-class support for missing values across the entire language
  • Literate programming support (like R-Markdown)
• Built-in Dates package that makes working with dates pleasant
• Ability to directly call and use R and Python code/packages with the PythonCall.jl and RCall packages
• Error messages are helpful and tell you what line the error came from, not just the type of error
• Debugger functionality so you can step through your code line by line

For power-users, advanced features are easily accessible: parallel programming, broadcasting, types, interfaces, metaprogramming, and more.

These are some of the things that make Julia one of the world’s most loved languages on the StackOverflow Developer Survey.

In addition to the liberal licensing mentioned above, there are professional products from organizations like JuliaHub that provide hands-on support, training, IT governance solutions, behind-the-firewall package management, and deployment/scaling assistance.

2.7 Tradeoffs when Using Julia

2.7.1 Just-In-Time Compilation

Julia is fast because it’s compiled, unlike R and Python where (loosely speaking) the computer just reads one line at a time. Julia compiles code “just-in-time” (JIT): right before you use a function for the first time, it will take a moment to pre-process the code section for the machine³. Subsequent calls don’t need to be re-compiled and are very fast.

A hypothetical example: running 10,000 stochastic projections where Julia needs to pre-compile but then runs each 10x faster:

• Julia runs in 2 minutes: the first projection takes 1 second to compile and run, but each 9,999 remaining projections only take 10ms.
• Python runs in 17 minutes: 100ms of a second for each computation.

Typically, the compilation is very fast (milliseconds), but in the most complicated cases it can be several seconds. One of these is the “time-to-first-plot” issue because it’s the most common one users encounter: super-flexible plotting libraries have a lot of things to pre-compile. So in the case of plotting, it can take several seconds to display the first plot after starting Julia, but then it’s remarkably quick and easy to create an animation of your model results. The time-to-first plot is a solvable problem that’s receiving a lot of attention from the core developers and will get better with future Julia releases.

For users working with a lot of data or complex calculations (like actuaries!), the runtime speedup is worth a few seconds at the start.

2.7.2 Static Binaries

Static binaries are self contained executable programs which can run very specific bits of code. Simple programs which can compile down to small (in terms of size on disk) binaries which accomplish just their pre-programmed tasks. Another use case for this is to create shared libraries which could be called from other languages (this can already be done, but again requires bundling the runtime).

Julia’s dynamic nature means that it needs to include the supporting infrastructure in order to run general code, similar to how running Python code needs to be bundled with the Python runtime.

Development is happening at the language level which would allow Julia to be compiled to a smaller, more static set of features for use in environments which are memory constrained and can’t bundle a supporting runtime.

2.8 Package Ecosystem

Using packages as dependencies in your project is assisted by Julia’ bundled package manager.

For each project, you can track the exact set of dependencies and replicate the code/process on another machine or another time. In R or Python, dependency management is notoriously difficult and it’s one of the things that the Julia creators wanted to fix from the start.

There are thousands of publicly available packages already published. It’s also straightforward to share privately, such as proprietary packages hosted internally behind a firewall.

Another powerful aspect of the package ecosystem is that due to the language design, packages can be combined/extended in ways that are difficult for other common languages. This means that Julia packages often interoperable without any additional coordination.

For example, packages that operate on data tables work without issue together in Julia. In R/Python, many features tend to come bundled in a giant singular package like Python’s Pandas which has Input/Output, Date manipulation, plotting, resampling, and more. There’s a new Consortium for Python Data API Standards which seeks to harmonize the different packages in Python to make them more consistent (R’s Tidyverse plays a similar role in coordinating their subset of the package ecosystem).

In Julia, packages tend to be more plug-and-play. For example, every time you want to load a CSV you might not want to transform the data into a dataframe (maybe you want a matrix or a plot instead). To load data into a dataframe, in Julia the practice is to use both the CSV and DataFrames packages, which help separate concerns. Some users may prefer the Python/R approach of less modular but more all-inclusive packages.

2.9 Tools in Your Toolbox

Looking at other great tools like R and Python, it can be difficult to summarize a single reason to motivate a switch to Julia, but hopefully we have sufficiently piqued your interest and we can turn to introducing important concepts.

That said, Julia shouldn’t be the only tool in your tool-kit. SQL will remain an important way to interact with databases. R and Python aren’t going anywhere in the short term and will always offer a different perspective on things!

Being a productive financial profession means being proficient in the language of computers so that you could build and implement great things. In a large way, the choice of tools and paradigms shape your focus. Productivity is one aspect, expressiveness is another, speed one more. There are many reasons to think about what tools you use and trying out different ones is probably the best way to find what works best for you.

---

1. Python first appeared in 1990. R is an implementation of S, which was created in 1976, though depending on when you want to place the start of an independent R project varies (1993, 1995, and 2000 are alternate dates). The history of these languages is long and substantial changes have occurred since these dates.

2. Aviva Case Study

3. Julia can also precompile code ahead of time to avoid the latency involved with running a function for the first time each time a Julia session is started.