Using Windows Subsystem for Linux to improve Django development

by | May 8, 2020

If you’re anything like me, you love using Linux as a development platform for Django. Nearly all webservers run Linux, the bash shell is awesome and developing on one platform only to deploy on another can sometimes bring up challenges like library compatibility, particularly when it comes to geospatial development that involves Geopandas or using graphical libraries like Cairo. Like many developers I have a desktop in my office and a laptop I take with me on the road since I work out of town frequently. Git is a great way to keep the project synchronized but the development environment must be compatible in order to effectively develop the web application.

Common issues I run into developing on Windows

Most of my Django development is creating geospatial applications. Inevitably this means I need to use the GDAL library which is simple in Linux but can be a major pain in a Windows environment. It is cross platform, and it can be installed in Windows but it behaves differently than in Linux.

If you’ve ever had this error you have probably gone down the rabbit hole of installing GDAL through OSGEO and either setting environment variables to the library path or added the following code to your settings.py file:

if os.name == ‘nt’:

import platform

OSGEO4W = r”C:\OSGeo4W”

if ’64’ in platform.architecture()[0]:

OSGEO4W += “64”

assert os.path.isdir(OSGEO4W), “Directory does not exist: ” + OSGEO4W

os.environ[‘OSGEO4W_ROOT’] = OSGEO4W

os.environ[‘GDAL_DATA’] = OSGEO4W + r”\share\epsg_csv”

os.environ[‘PROJ_LIB’] = OSGEO4W + r”\share\proj”

GDAL_LIBRARY_PATH = r’C:\OSGeo4W64\bin\gdal204′

os.environ[‘PATH’] = OSGEO4W + r”\bin;” + os.environ[‘PATH’]

Adding that code works just fine as long as you have used OSGEO to install the GDAL libraries, however some further modification of the libgdal.py file is likely necessary if you’ve installed a version of GDAL that is newer than 2.3.x as Django’s GIS package is set to look for the gdal203.dll and lower files. This isn’t a major issue on a single computer but adding that code is a bit messy when it will eventually be deployed on a Linux server anyway and it means that for any contributors to the project they must have the GDAL library installed in the exact same location or it will throw a file not found error. Finally, some people install the GDAL library separately to use with Geopandas and having multiple installations can sometimes cause conflicts, which means it is actually best to set the environment variables directly which does work but requires everyone to conform to that particular workflow.

Once libraries such as Weasyprint get involved to generate highly stylized PDF exports a whole extra host of issues pops up requiring the installation of MYSYS2 to install dependencies that simply exist in a Linux environment which again requires either setting environment variables through code or directly in the system.

Installing and enabling WSL is an easy solve

I’m going to argue that Windows Subsystem for Linux (WSL) is easier to install than scraping together all the dependencies you need for one main reason; it’s supported by Microsoft. Yes, this is going to get some hackles up but hear me out…

WSL keeps libraries and dependencies compartmentalized

As has been previously mentioned, specialized tools for open-source geospatial tools can be a bit fragmented. QGIS installs its own version of GDAL, PROJ and Python and it can be confusing as to which versions you need to install to get things running with GeoDjango. This can be problematic of frustrating particularly if you don’t need to use QGIS or any of the other OSGEO utilities for any other workflow.

Look at the image above, that is the OSGEO4W advanced installation window and it only shows SOME of the total GDAL packages you can install and doesn’t even touch on the PROJ library. If you don’t feel using the express install and installing PostgreSQL, QGIS and Apache on your development system you use this. For the average user, which GDAL packages do you need to install to work with your development environment? Wading through the myriad of online tutorials and conflicting information that is all-too-often out of date can be daunting. I have a Masters of GIS and it still can be a little overwhelming. We can’t all be experts in everything and quite often web developers would rather focus their efforts on developing using their chosen tools rather than figuring out how to install said tools to begin with.

Now, to get things running we have discussed the need to set environment variables using one method or another. This works fine if you only need to develop the web application. But having multiple versions of GDAL installed and registered in PATH and GDAL_VERSION etc. can cause conflicts when running other applications. For example, GeoPandas requires its own GDAL installation and can conflict with OSGEO4W at times if not set up properly. By using GeoDjango within a Linux installation through WSL you separate the dependencies while simultaneously making them easier to install.

WSL makes it easy to script installation and configuration for GeoDjango development

Once you have enabled WSL an installed your preferred Linux distribution from the Microsoft Store it is a simple process to install all of your desired packages. To get started with GeoDjango development you will need to install the following:

  • python3-pip
  • binutils
  • libproj-dev
  • gdal-bin
  • postgis (optional)

To do this simply run “bash” from the run dialog in Windows

Inside the bash terminal use apt install to install the necessary dependencies using the following command:

sudo apt install python3-pip binutils libproj-dev gdal-bin postgis

Once those are installed you can get to installing the necessary Python packages. I recommend installing virtualenv as it is always best to use virtual environments to prevent conflicts between packages in different projects but this is not strictly necessary. All of these are installed exactly as if you were working natively in your Linux distro of choice because…you are!

Document everything you install and add it to a text file or .sh file if you want so you can easily just copy and paste or run the shell script on new machines to get up and running fast on co-worker machines or on you mobile workstation.

Using your preferred IDE from within Windows and configuring it to work with WSL

Many common IDEs can use the Python interpreter installed inside your WSL Linux installation to run/debug your project. I’ll briefly cover two of the ones I frequently work with.

Creating a Virtualenv in WSL to use

Launch your WSL shell and create a virtual environment for Python exactly as you would working in Linux natively. I’m going to create one in my home folder under demoenv that we can then use in Pycharm and VS Code.

Visual Studio Code

Microsoft has excellent documentation here: Using Visual Studio Code with WSL for Web development.

Visual Studio Code is a free IDE that supports Django and is fully integrated with WSL. It is not my preferred IDE but if you want a free solution that allows full integration with WSL this is the one I would choose. Simply install Visual Studio Code and launch the bash shell. Create a directory to store your projects in and then create a virtualenv for your project. Activate your virtualenv, create your project and apps as you would normally and test it by running manage.py runserver. You will be able to access the Django development server directly from Windows.

To open the project in Visual Studio Code just navigate to the project folder in bash and type the following command:

code .

That’s it, you can develop directly in Visual Studio Code using the virtualenv you created in Linux.

Pycharm

Pycharm is by far my favorite IDE. However, the free Community version does not support WSL. Upgrading to the Professional version adds not only WSL support but much improved support for web development with Django as well as CSS and HTML hinting.

It is really as simple as opening a new project either using the Windows system python interpreter as the default or creating a new virtualenv based off the Windows interpreter. Then once you have created the Pycharm project you can set the WSL interpreter as the default interpreter for both the debugger and terminal within PyCharm.

Despite what some documentation says you CAN use Virtual Environments created in the WSL environment. Simply add a new interpreter and instead of using the default WSL python path change it to your virtualenv path. You can follow along in the video below.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.