The test suite depends on tox and pytest:
pip install tox pytest
To run the tests for all supported Python versions:
If you want to test only a specific Python version (e.g. Python 2.7), it’s as easy as:
tox -e py27
Tests are also run automatically on Travis CI.
You want to add a test for Jedi? Great! We love that. Normally you should write your tests as Blackbox Tests. Most tests would fit right in there.
For specific API testing we’re using simple unit tests, with a focus on a simple and readable testing structure.
Jedi is mostly being tested by what I would call “Blackbox Tests”. These tests are just testing the interface and do input/output testing. This makes a lot of sense for Jedi. Jedi supports so many different code structures, that it is just stupid to write 200‘000 unittests in the manner of regression.py. Also, it is impossible to do doctests/unittests on most of the internal data structures. That’s why Jedi uses mostly these kind of tests.
There are different kind of tests:
Integration test cases are located in test/completion directory and each test cases are indicated by the comment #? (completions / definitions), #! (assignments) and #< (usages). There is also support for third party libraries. In a normal test run they are not being executed, you have to provide a --thirdparty option.
In addition to standard -k and -m options in py.test, you can use -T (–test-files) option to specify integration test cases to run. It takes the format of FILE_NAME[:LINE[,LINE[,...]]] where FILE_NAME is a file in test/completion and LINE is a line number of the test comment. Here is some recipes:
Run tests only in basic.py and imports.py:
py.test test/test_integration.py -T basic.py -T imports.py
Run test at line 4, 6, and 8 in basic.py:
py.test test/test_integration.py -T basic.py:4,6,8
See py.test --help for more information.
If you want to debug a test, just use the --pdb option.
If you don’t like the output of py.test, there’s an alternate test runner that you can start by running ./run.py. The above example could be run by:
./run.py basic 4 6 8
The advantage of this runner is simplicity and more customized error reports. Using both runners will help you to have a quicker overview of what’s happening.
Uses comments to specify a test in the next line. The comment says, which results are expected. The comment always begins with #?. The last row symbolizes the cursor.
#? ['real'] a = 3; a.rea
Because it follows a.rea and a is an int, which has a real property.
Definition tests use the same symbols like completion tests. This is possible because the completion tests are defined with a list:
#? int() ab = 3; ab
Tests look like this:
abc = 1 #! ['abc=1'] abc
Additionally it is possible to add a number which describes to position of the test (otherwise it’s just end of line):
#! 2 ['abc=1'] abc
Tests look like this:
abc = 1 #< abc@1,0 abc@3,0 abc
Refactoring tests work a little bit similar to Black Box tests. But the idea is here to compare two versions of code. Note: Refactoring is currently not in active development (and was never stable), the tests are therefore not really valuable - just ignore them.