Fork me on GitHub

Jedi Testing

The test suite depends on tox and pytest:

pip install tox pytest

To run the tests for all supported Python versions:

tox

If you want to test only a specific Python version (e.g. Python 2.7), it’s as easy as:

tox -e py27

Tests are also run automatically on Travis CI.

You want to add a test for Jedi? Great! We love that. Normally you should write your tests as Blackbox Tests. Most tests would fit right in there.

For specific API testing we’re using simple unit tests, with a focus on a simple and readable testing structure.

Blackbox Tests (run.py)

Jedi is mostly being tested by what I would call “Blackbox Tests”. These tests are just testing the interface and do input/output testing. This makes a lot of sense for Jedi. Jedi supports so many different code structures, that it is just stupid to write 200‘000 unittests in the manner of regression.py. Also, it is impossible to do doctests/unittests on most of the internal data structures. That’s why Jedi uses mostly these kind of tests.

There are different kind of tests:

  • completions / goto_definitions #?
  • goto_assignments: #!
  • usages: #<

How to run tests?

Jedi uses pytest to run unit and integration tests. To run tests, simply run py.test. You can also use tox to run tests for multiple Python versions.

Integration test cases are located in test/completion directory and each test cases are indicated by the comment #? (completions / definitions), #! (assignments) and #< (usages). There is also support for third party libraries. In a normal test run they are not being executed, you have to provide a --thirdparty option.

In addition to standard -k and -m options in py.test, you can use -T (–test-files) option to specify integration test cases to run. It takes the format of FILE_NAME[:LINE[,LINE[,...]]] where FILE_NAME is a file in test/completion and LINE is a line number of the test comment. Here is some recipes:

Run tests only in basic.py and imports.py:

py.test test/test_integration.py -T basic.py -T imports.py

Run test at line 4, 6, and 8 in basic.py:

py.test test/test_integration.py -T basic.py:4,6,8

See py.test --help for more information.

If you want to debug a test, just use the --pdb option.

Alternate Test Runner

If you don’t like the output of py.test, there’s an alternate test runner that you can start by running ./run.py. The above example could be run by:

./run.py basic 4 6 8

The advantage of this runner is simplicity and more customized error reports. Using both runners will help you to have a quicker overview of what’s happening.

Auto-Completion

Uses comments to specify a test in the next line. The comment says, which results are expected. The comment always begins with #?. The last row symbolizes the cursor.

For example:

#? ['real']
a = 3; a.rea

Because it follows a.rea and a is an int, which has a real property.

Goto Definitions

Definition tests use the same symbols like completion tests. This is possible because the completion tests are defined with a list:

#? int()
ab = 3; ab

Goto Assignments

Tests look like this:

abc = 1
#! ['abc=1']
abc

Additionally it is possible to add a number which describes to position of the test (otherwise it’s just end of line):

#! 2 ['abc=1']
abc

Usages

Tests look like this:

abc = 1
#< abc@1,0 abc@3,0
abc

Refactoring Tests (refactor.py)

Refactoring tests work a little bit similar to Black Box tests. But the idea is here to compare two versions of code. Note: Refactoring is currently not in active development (and was never stable), the tests are therefore not really valuable - just ignore them.