A brief overview

The Jacobi method relies on the additive decomposition:

where is the diagonal component of , and consists of the off-diagonal terms. Plugging this decomposition into our system leads to

Starting from an initial guess , the core idea of the Jacobi method is to treat the diagonal contributions implicitly and the off-diagonal ones explicitly, analoguous to a time-integration scheme. This leads to the following iterative scheme

where subscript denotes the -th iteration of the method. What we claim then is that, under suitable assumptions on , the iterate converges to the actual solution of the system as . So when does it converge? And if so, how fast does it converge?¹

Fair enough, but does it actually converge?

The questions of whether or not an iterative method converges and, if so, how fast does it converge are obviously critical to assess its competitiveness. To answer to both of these questions, let us rewrite the Jacobi iteration as

To derive this expression, simply add and subtract inside the parenthesized term in the right-hand side and group terms together. Now, let be the true solution of the system, i.e. , and be the error at iteration . Using simple algebraic manipulations, the dynamics of the error vector are governed by

Obviously, the Jacobi method converges to the correct solution provided where is a suitable vector norm. The question of its convergence thus reduces to: under what condition on does the norm of the error vector goes to zero?

Theorem n°1 – The Jacobi iterative method

converges for any initial vector provided where is a matrix norm induced by the corresponding vector norm.

Sketch of the proof – Let be a matrix norm consistent with a vector norm. Then

A simple inductive argument shows that (for large enough)

Hence, converges to zero as for all provided that .

Alright, we now know the Jacobi method converges provided . But what are the necessary and/or sufficient conditions on for to be less than unity?

Theorem n°2 – Let and be symmetric positive definite matrices. Then, the Jacobi iteration converges.

The proof is divided in two parts.

Proof (part 1) – Let be symmetric positive definite and be an eigenvalue of with eigenvector . Then Mutliplying from the left by leads to Then Re-arranging terms yields and being symmetric positive definite, we have It implies and thus . Hence, all the eigenvalues of are less than unity.

While we arrived at the conclusion that , nothing so far implies and thus . This is where the condition on comes into play.

Proof (part 2) – Let be symmetric positive definite. Then and thus From part 1, we know that . Hence implying . Combined with part 1, we thus have , i.e. the eigenvalues of are inside the unit circle (and real) and the Jacobi iteration converges.

Note that the condition “ and being symmetric positive definite” is sufficient although not necessary to guarantee the convergence of the Jacobi method. Another classical sufficient but non-necessary condition is that is strictly row diagonally dominant. Again, it is relatively easy to prove but since I’m the teacher here, I’ll end this theoretical analysis with the nefarious: This is left as an exercise for the reader.

Alright, it converges. But how fast?

We’ve actually already partially answered this question. From the sketch of the proof for Theorem n°1, we have

Obviously, the smaller , the faster the convergence. And we know that is related to the eigenvalues of the iteration matrix being inside the unit circle. So how do the eigenvalues influence the convergence rate of the method?

A useful quantity to estimate the convergence rate of the method is the spectral radius of the iteration matrix . It is defined as

where are the eigenvalues. Moreover, for every natural matrix norms. From our previous discussion, we know that since the Jacobi method converges. Eventhough the spectral radius is only a lower bound for , we’ll assume for the sake of simplicity that it is pretty tight. Hence, we roughly have

We could make this statement more formal but it wouldn’t change the intuition: the smaller the spectral radius, the larger the asymptotic convergence rate. Unfortunately, there is not much else to say without knowing exactly the matrix so let’s turn to the actual problem of interest of this post.

Discretizing the problem

There are many different ways to discretize a partial differential equation. Finite differences, finite volumes, finite elements, spectral elements, spectral methods, pseudo-spectral methods, etc. There are no silver bullets though. Each has its pros and cons. At the end of the day, the discretization method used is often a matter of personal preferences. To keep things simple, we will consider the standard second-order accurate finite-difference scheme. We will also consider a uniform grid spacing in each direction so that our differential operators can be approximated as

where and are the grid sizes in each direction, and is the value of our unknown function evaluated at the grid point . For the sake of simplicity, we’ll assume furthermore that . Our discretized partial differential equation for points inside of the domain then reads

It may not seem like a linear system, but trust me, it is. The field is represented as a two-dimensional array (and I mean array, not matrix) for the sake of simplicity. But you can always represent it as a vector by simply stacking the columns of the array on top of one another, and likewise for the right-hand side forcing .

So, where is the matrix then? Consider a single column of the array , that is we fix and only consider different -values. The second-order derivative in the -direction can be represented as an matrix given by

where we excluded the points on the upper and lower boundaries as these are equal to zero owing to our choice of boundary conditions. Likewise, considering a single row of (i.e. fixing and considering different -values), the second-order derivative in the horizontal direction can be represented as an matrix with the same tridiagonal structure as . Our problem can then be represented in a standard linear system form as

where is the Kronecker product. If we were to explicitly construct it, this matrix would have rows and likewise for the number of columns. For a discretization employing 512 points in each direction, that would be 260 100 columns and rows. Pretty big then, and completely intractable for standard direct linear solvers!

The Jacobi method for the 2D Poisson equation

Time to write the Jacobi update rule for our particular problem. Recall that our problem reads

On the left-hand side, the term corresponds to the diagonal component while all the others are the off-diagonal ones. Following what we have written for the Jacobi method in matrix form, specializing for this equation leads to the following update rule

where the superscript denotes the iteration number. This will be fairly simple to implement. Create two arrays, one to store the solution at iteration and the other one at iteration . Loop over the indices and update the -th entries of the second table with the appropriate combination of values from the first one. Note that it is important to keep these two tables. If you were to have only one table and directly update its entry, you would end-up with a different method: Gauss-Seidel. More on that in a later post (maybe).

Convergence properties – The second-order accurate central finite-difference approximation of the Laplace operator is a symmetric negative definite matrix. Hence, is symmetric positive definite (and this is the reason for why there is a minus sign on the right-hand side of our problem if you wondered). It is easy to show moreover that satisfies the assumptions for Theorem n°2 to hold. Hence, after a sufficiently large number of iterations, the Jacobi method will converge to the actual solution of our linear system. But again, how fast?

As before, we can get some intuition by looking at the spectral radius of this matrix. I won’t go through the calculations (and I’ll assume ), but we basically have

Using an increasing number of grid points to discretize our domain (that is considering a finer and finer mesh), the spectral radius of the iteration matrix gets closer and closer to unity. As a consequence, the Jacobi method requires more and more iterations to compute a reasonnably accurate solution. This poor scaling property is one of the reasons why it ain’t actually used nowadays in high-performance computing solvers. But this does not concern us here.

Baseline implementation

Let us start with an almost verbatim translation of the pseudo-code to Fortran. In the rest, we will use double precision arithmetic. The kind parameter will be defined as

integer, parameter :: dp = selected_real_kind(15, 307)

This is often considered to be a good practice in Fortran and guarantees a certain portability of the code across different compilers and platforms. Let us now turn our attention to the Jacobi kernel. Our textbook implementation is shown below.

pure subroutine textbook_kernel(nx, ny, u, v, b, dx)
    implicit none
    integer, intent(in) :: nx, ny
    real(dp), intent(out) :: u(nx, ny)
    real(dp), intent(in) :: v(nx, ny), b(nx, ny), dx
    integer :: i, j
    do j = 2, ny-1
        do i = 2, nx-1
            u(i, j) = 0.25_dp*(b(i, j)*dx**2 - v(i+1, j) - v(i-1, j) &
                                             - v(i, j+1) - v(i, j-1))
        enddo
    enddo
end subroutine

The pure keyword is here to tell the compiler that we guarantee this subroutine has no unintended side-effect. It is not technically mandatory, but it is also part of the good practices in Fortran. Hopefully, if we do things right, this kernel should be where we spend most of the computational time. To the actual solver now.

function textbook_solver(b, tol, maxiter) result(u)
    implicit none
    real(dp), intent(in) :: b(:, :), tol
    integer, intent(in) :: maxiter
    real(dp), allocatable :: u(:, :)
    ! Internal variables
    integer :: nx, ny, i, j, iteration
    real(dp), allocatable :: v(:, :)
    real(dp) :: dx, l2_norm
    ! Initialize variables
    nx = size(b, 1); ny = size(b, 2); dx = 1.0_dp/(nx - 1)
    if (nx /= ny) then
        error stop "Number of points in each direction need to be equal."
    endif
    allocate (u(nx, ny), source=0.0_dp)
    allocate (v(nx, ny), source=0.0_dp)
    l2_norm = 1.0_dp
    iteration = 0
    ! Begining of the Jacobi iterative method.
    do while ((iteration < maxiter) .and. (l2_norm > tol))
        ! Jacobi iteration.
        call textbook_kernel(nx, ny, v, u, b, dx)
        ! Compute error norm.
        l2_norm = norm2(u - v)
        ! Update variable.
        u = v
        ! Update iteration counter.
        iteration = iteration + 1
    end do
    print *, "Textbook solver :"
    print *, "    - Number of iterations :", iteration
    print *, "    - l2-norm of the error :", l2_norm
end function

Even if you ain’t familiar with Fortran, the code should be quite readable. After having declared and initialized all of the required variables, the Jacobi method starts from Line 20 and proceeds in 3 steps:

1. Perform the Jacobi update by calling our textbook kernel.
2. Compute the 2-norm of the correction.
3. Update the current solution with its latest estimate.

This loop keeps on going until the 2-norm of the correction is small enough to claim convergence. In all of our experiments, the tolerance is set to .

Performances – We will use 512 points in each direction with a uniform grid spacing and assume the initial guess to be the zero solution for all of our experiments. We thus have slightly more than a quarter million of unknowns, a reasonnably large linear system. The code is compiled using gfortran 15.1 and the following options: -O3 -march=native -mtune=native. The table below summarizes some of the key computational metrics.

Solving a linear system with a quarter million of unknowns in under one minute is quite impressive when you think about it. It is clearly orders of magnitude faster than if you were to do it by hand (and far less error-prone)! But is this the best we can do? You might be inclined to say yes. After all, our implementation is an almost verbatim translation of the pseudo-code and maths don’t lie. But that ain’t completely true though… When it comes to scientific computing, there are many streetfighting skills you can pick along the way to massively improve the computational performances of a given algorithm. So let’s start optimizing!

You shall not copy!

Our Jacobi kernel is so simple that there ain’t much room for improvement so let’s look at the solver itself starting with line 26

    u = v

It is the update of our current estimate of the solution with the one we’ve just computed. It essentially is a copy operation. Given how simple our Jacobi kernel is, it acutally takes almost as long as computing a Jacobi update. So let’s get rid of it by simply performing an additional call to the Jacobi kernel with the role of u and v being flipped.

function nocopy_solver(b, tol, maxiter) result(u)
    implicit none
    real(dp), intent(in) :: b(:, :), tol
    integer, intent(in) :: maxiter
    real(dp), allocatable :: u(:, :)
    ! Internal variables
    integer :: nx, ny, i, j, iteration
    real(dp), allocatable :: v(:, :)
    real(dp) :: dx, l2_norm
    ! Initialize variables
    nx = size(b, 1); ny = size(b, 2); dx = 1.0_dp/(nx - 1)
    if (nx /= ny) then
        error stop "Number of points in each direction need to be equal."
    endif
    allocate (u(nx, ny), source=0.0_dp)
    allocate (v(nx, ny), source=0.0_dp)
    l2_norm = 1.0_dp
    iteration = 0
    ! Begining of the Jacobi iterative method.
    do while ((iteration < maxiter) .and. (l2_norm > tol))
        ! Jacobi iteration (no copy).
        call textbook_kernel(nx, ny, v, u, b, dx)
        call textbook_kernel(nx, ny, u, v, b, dx)
        ! Compute error norm.
        l2_norm = norm2(u - v)
        ! Update iteration counter.
        iteration = iteration + 2
    end do
    print *, "No-copy solver  :"
    print *, "    - Number of iterations :", iteration
    print *, "    - l2-norm of the error :", l2_norm
end function

Performances – Code-wise, very little has changed compared to our baseline implementation. The nocopy_solver slightly departs from the pseudo-code but is still as readable. Peformance-wise, it is a different story.

This one-line change makes our solver compute the solution twice as fast! But don’t get too excited, it is somewhat expected if you think of it. A copy is roughly as expensive as computing a Jacobi update itself. As a consequence, in the time frame it took our baseline implementation to peform a Jacobi update followed by a copy, the nocopy_solver performed no copy (hence the name) but two updates. And boom, twice as fast. This is the first but probably most important take-away message:

Avoid copies like the plague and re-use intermediate results as much as possible.

This is not specific to Fortran and is true for pretty much any programming language you use.

Further optimizations

While the no-copy trick is fairly general, let us turn now to somewhat Jacobi-specific optimization tricks starting with the Jacobi kernel itself. Let be the current approximate solution and the new one being computed. Recall that the update rule is as follows

for all and corresponding to points in the interior of the computational domain. One crucial observation is that there are no dependencies between the entries of : they can be update in any abitrary order, not necessarily the lexicographic one. In particular, we could let the compiler decide on its own what is the most efficient way to do this update based on its internal mechanics. The 2008 standard introduced a particular construct conveying precisely this: the do concurrent. Below is the Jacobi kernel rewritten using this construct.

pure subroutine doconcurrent_kernel(nx, ny, u, v, b, dx)
    implicit none(external)
    integer(ilp), intent(in) :: nx, ny
    real(dp), intent(out) :: u(nx, ny)
    real(dp), intent(in) :: v(nx, ny), b(nx, ny), dx
    integer :: i, j
    do concurrent(i=2:nx - 1, j=2:ny - 1)
        ! Jacobi update.
        u(i, j) = 0.25_dp*(b(i, j)*dx**2 - v(i + 1, j) - v(i - 1, j) &
                                         - v(i, j + 1) - v(i, j - 1))
    end do
end subroutine doconcurrent_kernel

For now, it does not actually improve the computational performances of our kernel. For serial computations, it mostly is a syntactic sugar letting someone reading the code know that this loop could technically be computed in parallel with no problem. It might help the compiler optimize a bit, but the kernel being so simple I haven’t seen much changes. It’ll be different though once we go to multithreaded computations but that’s a story for slightly later.

The main source of computational improvement is located on line 25:

    l2_norm = norm2(u-v)

There is nothing particularly wrong with this line. In practice however, the Jacobi method is quite slow to converge and computing the residual norm at every iteration incurs extra computational costs which are unecessary. We would be much better off by checking the residual only once in a while. We could replace it with

    if (mod(iteration, 1000)) l2_norm = norm2(u - v)

or using the newest do concurrent construct

if (mod(iteration, 1000)) then
    l2_norm = 0.0_dp
    do concurrent(i=2:nx-1, j=2:ny-1) reduce(+:l2_norm)
        l2_norm = l2_norm + (u(i, j) - v(i, j))**2
    enddo
    l2_norm = sqrt(l2_norm)
endif

Either way is fine, norm2 is an intrinsic Fortran function and its implementation has already been optimized by the compiler vendors anyway. Checking the residual norm every 1000 iterations is arbitrary. It has been chosen out of simplicity considering that the method takes 128 000 iterations to converge for our particular problem. In practice, you might actually pass this as an extra argument to the solver to let the user decide. Here is the updated solver.

function doconcurrent_solver(b, tol, maxiter) result(u)
    implicit none(external)
    real(dp), intent(in) :: b(:, :), tol
    integer, intent(in) :: maxiter
    real(dp), allocatable :: u(:, :)
    ! Internal variables.
    integer :: nx, ny, i, j, iteration
    real(dp), allocatable :: v(:, :)
    real(dp) :: dx, l2_norm
    ! Initialize variables
    nx = size(b, 1); ny = size(b, 2); dx = 1.0_dp/(nx - 1)
    if (nx /= ny) then
        error stop "Number of points in each direction need to be equal."
    endif
    allocate (u(nx, ny), source=0.0_dp)
    allocate (v(nx, ny), source=0.0_dp)
    l2_norm = 1.0_dp
    iteration = 0
    do while ((iteration < maxiter) .and. (l2_norm > tol))
        ! Jacobi kernel (no-copy).
        call doconcurrent_kernel(nx, ny, v, u, b, dx)
        call doconcurrent_kernel(nx, ny, u, v, b, dx)
        ! Compute error norm.
        if (mod(iteration, 1000) == 0) l2_norm = error_norm(u, v)
        ! Update iteration counter.
        iteration = iteration + 2
    end do
    l2_norm = error_norm(u, v) ! Sanity check
    print *, "Do-concurrent solver :"
    print *, "    - Number of iterations :", iteration
    print *, "    - l2-norm of the error :", l2_norm
end function

pure real(dp) function error_norm(u, v) result(l2_norm)
    implicit none
    real(dp), intent(in) :: u(:, :), v(:, :)
    integer :: i, j
    l2_norm = 0.0_dp
    do concurrent(i=2:size(u, 1)-1, j=2:size(u, 2)-1) reduce(+:l2_norm)
        l2_norm = l2_norm + (u(i, j) - v(i, j))**2
    end do
    l2_norm = sqrt(l2_norm)
end function

Performances – Again, the new solver is just as readable as the previous ones. No big changes here, but look at the performances below!

The new solver is 3 to 4 times faster than our baseline! This is quite remarkable given that we changed only a couple of lines compared to the original textbook implementation. Things are not always so clear cut for more complex algorithms, but still. And here is our second take-way:

Compute just what you need, not more.

Here, computing the residual norm for each iteration was clearly a non-negligible and non-necessary bottleneck. At this point, there is no more low-hanging fruit for optimization. You might think this is it. A 3.6x speed-up is good enough and you may call it a day. But you know what? There’s more. We can reach a 20x to 30x speed-up without changing anything else to the code!

Multithreaded performances

Computers these days tend to have built-in parallel computing capabilities. Yet, we haven’t leverage these so far. Let’s change that. I will not get into a discussion about openMP vs MPI or GPU offloading. I will keep things very practical instead. Remember when I said we can perform the Jacobi update in any order we want? The Jacobi update rule is embarassingly parallel. That is precisely what the do concurrent construct is conveying. And compilers can leverage this for increased computational performances. For our solver, it is as simple as changing the gfortran compilation options from

-O3 -mtune=native -march=native

to

-O3 -mtune=native -march=native -ftree-parallelize-loops=n

where n is the number of processes/threads to be used. And that’s it. Litterally. And look at these performances!

As promised, we finish with a linear solver computing the solution of a system with a quarter million of unknowns in less than 3 seconds and not a single openMP pragma or MPI call. The code is the exact same as before. The only thing that changed is the addition of the new compilation option. And that is enough to reach a 27x speed-up compared to our original textbook implementation!² Pretty good considering we changed only a handful of lines of code and added only one extra compilation option, innit? There would be a lot more to say about the different parallel computing paradigms and the associated neaty greedy details, but this post is already sufficiently long as it is so I’ll stop right there. I’ll leave you with a cautionnary quote by the famous Donald Knuth though

The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming.

Where do students struggle?

Admittidely, the code is quite readable and look very similar to the pseudocode you’d use to describe the Jacobi method. But if you’re reading this blog post, there probably are a handful of things you’ve internalized and don’t even think about anymore (true for both Python and Fortran). And that’s precisely what the students (at least mine) struggle with, starting with the very first line.

What the hell is numpy and why do I need it? Also, why import it as np? – These questions come back every year. Yet, I don’t have satisfying answers. I always hesitate between

Trust me kid, you don’t want to use nested lists in Python to do any serious numerical computing.

which naturally begs the question of why, or

When I said we’ll use Python for this scientific computing class, what I really meant is we’ll use numpy which is a package written for numerical computing because Python doesn’t naturally have good capabilities for number crunching. As for the import as np, that’s just a convention.

And this naturally leads to the question of “why Python in the first place then?” for which the only valid answer I have is

Well, because Python is supposed to be easy to learn and everybody uses it.

Clearly, import numpy as np is an innocent-looking line of code. It has nothing to do with the subject being taught though, and everything with the choice of the language, only diverting the students from the learning process.

I coded everything correctly, 100% sure, but I get this weird error message about indentation – Oh boy! What a classic! The error message varies between

IndentationError: expected an indented block

and

TabError: inconsistent use of tabs and spaces in indentation

<TAB> versus SPACE is a surprisingly hot topic in programming which I don’t want to engage in. A seasoned programmer might say “simply configure your IDE properly” which is fair. But we’re talking about your average student (who’s not a CS one remember) and they might use IDLE or even just notepad. As for the IndentationError, it is a relatively easy error to catch. Yet, the fact that for, if or while constructs are not clearly delineated in Python other than visually is surprisingly hard for students. I find that it puts an additional cognitive burden on top of a subject which is already demanding enough.

It could also be more subtle. The code might run but the results are garbage because the student wrote something like

    for iteration in range(maxiter):
    # Jacobi iteration.
    for i in range(1, nx-1):
    for j in range(1, ny-1):
    tmp[i, j] = 0.25*(b[i, j]*dx**2 - u[i+1, j] - u[i-1, j] 
                                                - u[i, j+1] - u[i, j-1])

You might argue that this perfectly understandable, though if you want to be picky, there is no dealineation of where the different loops end. Which the whole point of indentation in Python. But students do not necessarily get that.

Why range(1, nx-1) and not range(2, nx-1)? The first column/row is my boundary. – Another classic related to 0-based vs 1-based indexing. And another very hot debate I don’t want to engage in. The fact however is that linear algebra (and a lot of scientific computing for that matter) use 1-based indexing. Think about vectors or matrices. Almost every single maths books write them as

The upper left element has the (1, 1) index, not (0, 0). Why use a language with 0-based indexing for linear algebra other than putting an additional cognitive burden on the students learning the subject? This is a recipe for the nefarious off-by-one error. And these errors are sneaky. The code might run but produce incorrect results and it’s a nightmare for the students (or the poor TA helping them) to figure out why.

Why np.linalg.norm and not just norm or np.norm? – This is one is related to my first point. When you’re used to it, you no longer question it. But you don’t know students then and, once more, I don’t have a really clear answer other than

Well, linalg stand for linear algebra, and np.linalg is a collection of linear algebra related function. It is a submodule of numpy, the package I told you about before.

Grouping like-minded functionalities into a dedicated submodule is definitely good practice, no question there. Discussing the architecture of numpy makes a lot of sense when students have to do a big project involving numerical computing but not strictly speaking about numerical computing. On the other hand, when it is their first numerical computing class (and possibly first with Python) I find it distracting. Again, it’s not a big thing really but still. And then you have to explain why np.det and np.trace are not part of np.linalg…

Other common problems – There are other very common problems like using the wrong function or inconsistent use of lower- or upper-case for variables. Once you know Python is case-sensitive, this is mainly a concentration problem. No big deal there. But there is one last thing that tends to cause problems to distracted students and that has to do with the dynamic nature of Python. Nowhere in the code snippet is it clearly specified that b needs to be a two-dimensional np.array of real numbers nor that it shouldn’t be modified by the function. It is only implicit. And that can be a big problem for students when working with marginally more complicated algorithms. Sure enough, type annotation is a thing now in Python, but it still is pretty new and comparatively few people actually use them.

What about Fortran?

Alright, I’ve spent the last five minutes talking shit about Python but how does Fortran compare with it? Here is a typical implementation of the same function. I’ve actually digged it from my own set of archived homeworks I did 15+ years ago and hardly modified it.

function jacobi(b, dx, tol, maxiter) result(u)
    implicit none
    real, dimension(:, :), intent(in) :: b
    real, intent(in) :: dx, tol
    integer, intent(in) :: maxiter
    real, dimension(:, :), allocatable :: u
    ! Internal variables.
    real, dimension(:, :), allocatable :: tmp
    integer :: nx, ny, i, j, iteration

    ! Initialize variables.
    nx = size(b, 1) ; ny = size(b, 2)
    allocate(u(nx, ny), source = 0.0)
    residual = 1.0

    ! Jacobi solver.
    do iteration = 1, maxiter
        ! Jacobi iteration.
        do j = 2, ny-1
            do i = 2, nx-1
                tmp(i, j) = 0.25*(b(i, j)*dx**2 - u(i+1, j) - u(i-1, j) &
                                                - u(i, j+1) - u(i, j-1))
            enddo
        enddo

        ! Compute residual.
        residual = norm2(u - tmp)
        ! Update solution.
        u = tmp
        ! If convered, exit the loop.
        if (residual <= tol) exit
    enddo

end function

No surprise there. The task is sufficiently simple that both implementations are equally readable. If anything, the Fortran one is a bit more verbose. But in view of what I’ve just said about the Python code, I think it actually a good thing. Let me explain.

Definition of the variables – Fortran is a strongly typed language. Lines 2 to 8 are nothing but the definitions of the different variables used in the routine. While you might argue it’s a pain in the a** to write these, I think it can actually be very beneficial for students. Before even implementing the method, they have to clearly think about which variables are input, which are ouput, what are their types and dimensions. And to do so, they have to have at least a minimal understanding of the algorithm itself. Once it’s done, there are no more surprises (hopefully), and the contract between the code and the user is crystal clear. And more importantly, the effort put in clearly identifying the input and output of numerical algorithm usually pays off and leads to less error-prone process.

Begining and end of the constructs – Fortran uses the do/end do (or enddo) construct, clearly specifying where the loop starts where it ends. The indentation used in the code snippet really is just a matter of style. In constrast to Python, writing

    do j = 2, ny-1
    do i = 2, nx-1
    tmp(i, j) = 0.25*(b(i, j)*dx**2 - u(i+1, j) - u(i-1, j) &
                                    - u(i, j+1) - u(i, j-1))
    enddo
    enddo

does not make the code any less readable and wouldn’t change a dime in terms of computations. It’s a minor thing, fair enough. But it instantly get rid of the IndentationError or TabError which are puzzling students. I may be wrong, but I believe it actually reduces the cognitive load associated with the programming language and let the students focus on the actual numerical linear algebra task.

No off-by-one error – By default, Fortran uses a 1-based indexing. No off-by-one errors, period.

Intrinsic functions for basic scientific computations – While you have to use np.linalg.norm in Python to compute the norm of a vector, Fortran natively has the norm2 function for that. No external library required. If you want to be picky, you may say that norm2 is a weird name and that norm might be just fine.

Some quirks of Fortran – All is not perfect though, starting with Line 2 and the implicit none statement. This is a historical remnant which is considered good practice by modern Fortran standards but not actually needed. Students being students, they will more likely than not ask questions about it although it has nothing to do with the subject of the class itself. Admittidely, it can be a bit cumbersome to explicitely define all the integers you use even if it’s just for a one-time loop. Likewise, there is the question of real vs double precision vs real(wp) (where wp is yet another variable you’ve defined somewhere). I don’t think it matters too much though when learning the basics of numerical linear algebra algorithms, although it certainly does when you start discussing about precision and performances.