Category Archives: django

Be careful with django fixture files

Reading Time: 1 minute

TL;DR

In this post I will explain the issue that can happen during the importing django fixture files into testing database.

Django fixture files enable developer to prepare testing data. We usually put in those files data that is static and essential for data model. For example, list of countries.

The problem is that developer needs to put by hand id fields that must be unique and sometimes also present in other fixture file (related data).

Some context. In the early development, you have several developers working on the feature. They install database on their development machine. Now we have one testing database, and several development databases.

Developer imports other fixture files and creates fixture files related with his features. Now it needs to include his fixtures to source control. The easiest way is to use django command, dumpdata which dumps whole database model. His fixture files contain already imported data from other developers.

Development process continues, which means that data will be added to developer local databases.

It is time for another fixture dump, and import of that dump fails on testing databases.

The reason is that ids in fixture files from several developers are not in sync. Quick and dirty solution is to drop the database. Which is maybe applicable at the early stage of development and on test database. But when we have production data, dropping the database is not the solution.

Solution? All developers should import (and resolve any conflicts using plain sql) fixture files from version control  into their local development database before they do dumpdata with new fixture files.

In that way, version control becomes central point for all database fixture files.

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

How to get full stack trace during gunicorn exception investigation

Reading Time: 2 minutes

TL;DR

This blog post will explain how to get full exception stack trace during the gunicorn start up issue. gunicorn is python native application server for running django application.

So you changed something in your django settings.py configuration, and gunicorn failed to start. In my case, I did not get full exception stack trace, which means I did not know what was the root cause for this issue.

Traceback (most recent call last):

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 196, in run

web_1 |     self.sleep()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 346, in sleep

web_1 |     ready = select.select([self.PIPE[0]], [], [], 1.0)

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 231, in handle_chld

web_1 |     self.reap_workers()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 506, in reap_workers

web_1 |     raise HaltServer(reason, self.WORKER_BOOT_ERROR)

web_1 | gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

web_1 |

web_1 | During handling of the above exception, another exception occurred:

web_1 |

web_1 | Traceback (most recent call last):

web_1 |   File "/usr/local/bin/gunicorn", line 11, in <module>

web_1 |     sys.exit(run())

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/app/wsgiapp.py", line 74, in run

web_1 |     WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/app/base.py", line 192, in run

web_1 |     super(Application, self).run()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/app/base.py", line 72, in run

web_1 |     Arbiter(self).run()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 218, in run

web_1 |     self.halt(reason=inst.reason, exit_status=inst.exit_status)

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 331, in halt

web_1 |     self.stop()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 381, in stop

web_1 |     time.sleep(0.1)

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 231, in handle_chld

web_1 |     self.reap_workers()

web_1 |   File "/usr/local/lib/python3.4/site-packages/gunicorn/arbiter.py", line 506, in reap_workers

web_1 |     raise HaltServer(reason, self.WORKER_BOOT_ERROR)

web_1 | gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
 Google led me to this usefull blog post.
gunicorn slweb.wsgi:application --preload
 And official gunicorn documentation explains –preload option:
Load application code before the worker processes are forked.
Because first exception describes worker spawn failure, loading application before worker spawn may trigger more detailed exception.
And that was true:
django.core.exceptions.ImproperlyConfigured: 'oracle' isn't an available database backend.
Try using 'django.db.backends.XXX', where XXX is one of:

    'base', 'mysql', 'oracle', 'postgresql_psycopg2', 'sqlite3'
 The root cause was wrong name for oracle driver.
Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Django time machine

Reading Time: 1 minute

TL;DR

In my previous blog post, Simulating time in Ruby on Rails framework, I described how to travel in time in both directions by using Ruby on Rails console. In this post I will describe same feature for Django framework.

Here is example how to update in django shell user table last_login column, using Django active record classes. User is filtered using email column.

cd to_root_of_your_django_project
>python3 manage.py shell
>> from sl_models import user
>> user_instance = user.models.User.objects.filter(email="user_email value")
>> user_instance.values()
>> from datetime import timedelta
>> from datetime import date
>> user_instance.update(last_login=date.today() - timedelta(days=7)) //we are traveling by days, but it is also possible to travel by other time dimensions. Check python timedelta documentation
>>user_instance.values()

I wish you happy time traveling in Django!

 

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

One character to rule them all

Reading Time: 1 minute

TL;DR

In this post I will provide example how just one character can make a significant difference regarding security of Django web application.

The issue is sql injection. When I test for sql injections and I have access to client codebase (which can save significant amount of money for client), I first search code for using raw sql code. I am using simple unix utilities, less and grep:

grep -H -r 'what_you_search' * | less

In Django code system, you should search for raw function because it accepts for input raw sql.

You should learn what is proper way to send sql parameters to that function. For Django raw, this is proper way:

>>> lname = 'Doe'
>>> Person.objects.raw('SELECT * FROM myapp_person WHERE last_name = %s', [lname])

I searched the codebase, and found following:

>>> lname = 'Doe'
>>> Person.objects.raw('SELECT * FROM myapp_person WHERE last_name = %s' % lname)

Have you noticed the difference? % instead of ,

Here is how you can easily construct strings in Python (Django is Python framework):

"welcome sql injection %s" % hacker_string

This just replaces hacher_string with %s. And does not check hacker_string for possible sql code injection, which raw function does, but only when user input is send as raw function parameter, as explained in documentation.

%, one character to rule them all!

Facebooktwittergoogle_plusredditpinterestlinkedinmailby feather