An Intro to Unit Testing in Python

and
PyTest framework features

by

Arulalan.T
arulalant@gmail.com

Project Associate,
Centre for Atmospheric Sciences ,
Indian Institute of Technology Delhi.
18.04.2011

TableofContents
Unit testing in Python ...................................................................................................................................................1
1.1 Unit test code in unit-test framework...........................................................................................................1
1.2 Functional code in unit-test framework......................................................................................................3
1.3 Testing result in unit-test framework...........................................................................................................3
Pytest Framework for unit testing In Python .........................................................................................................4
2.1 How to install Pytest ........................................................................................................................................4
2.2 Unit test code in PyTest framework.............................................................................................................4
2.3 Testing result in PyTest framework..............................................................................................................5
Pytest Usages and Advantages for unit testing in Python .................................................................................6
3.1 Getting help on version, option names, environment variables...........................................................6
3.2 Stopping after the first (or N) failures.........................................................................................................6
3.3 Specifying tests / selecting tests....................................................................................................................6
3.4 Dropping to PDB (Python Debugger) on failures....................................................................................6
3.5 Setting a breakpoint / aka set_trace()...........................................................................................................6
3.6 creating resultlog format files........................................................................................................................6
Skipping and Xfail testes in Pytest ...........................................................................................................................7
4.1 Marking a test function to be skipped..........................................................................................................7
4.2 skip all test functions of a class.....................................................................................................................7
4.3 Mark a test function as expected to fail.......................................................................................................8
4.4 Evaluation of skipif/xfail expressions..........................................................................................................9
4.5 Imperative xfail from within a test or setup function..............................................................................9
4.6 Skipping on a missing import dependency................................................................................................9
4.7 Imperative skip from within a test or setup function............................................................................10
4.8 Using -k TEXT to select tests.......................................................................................................................10
Extended xUnit style setup fixtures in PyTest.....................................................................................................11
5.1 module level setup/teardown........................................................................................................................11
5.2 class level setup/teardown.............................................................................................................................11
5.3 method and function level setup/teardown...............................................................................................11
xdist: pytest distributed testing plugin....................................................................................................................13
6.1 Installation of xdist plugin.............................................................................................................................13
6.2 Usage examples................................................................................................................................................13
More Usage of PyTest.................................................................................................................................................14
7.1 Running tests in a Python subprocess........................................................................................................14
7.2 Running tests in looponfailing mode.........................................................................................................14
7.3 Sending tests to remote SSH accounts......................................................................................................14
7.4 Sending tests to remote Socket Servers.....................................................................................................15
7.5 Specifying test exec environments in an ini file.....................................................................................15
7.6 Specifying rsync dirs in an ini-file.........................................................................................................15

Links for pytest .............................................................................................................................................................16

Unit testing in Python

Tounderstandaboutunittestingbriefly,kindlystarttostudyitfromDiveintoPython/unit_testing.
ThisisveryshortnotesaboutanintrotounittestinginpythonandwhywechoosesPyunitasunittestframeworkin
ourproject.
Unittestmodulecomesinthedefaultpythonfrom2.2versionitself.Wenoneedtoinstallanythingforsimple
unittesting.
Letstakesimplemodfunctiongoingtobewrittenbythedeveloper.Forthatwehavetowritetheunittesting
codefirst.Thenonlythedevelopershouldstarttowritethecodingformodfunction.usuallyfunctiondeveloperand
unittestdevelopershouldbethesameperson.Somecasesitmaychange.
Ingeneralthequalityoffunctioncodetobechecked/passedbytheunittest.Ifboththeunittestingcodeand
functionalcodeshouldbewrittenbythesamedeveloper,thentheywilllovetowritetheunittesting.Unittestimproves
thequalityofthefunctionalcodeandmakeitasstandardone.!.
Phase0:
Thezerothphaseoftheunittestis,wehavetodecidewhatwearegoingtotest?
Inourexample,wearegoingtowritetheunittestformodfunctionwhichisthepartofthemodulecalleddivisible.py.

Modfunctionshouldreturnthecorrectvalue.Itmaycheckedbypassingtherandominputorknown
values.
ModfunctionshouldraisetheZerodenominatorerrorifuserwillpasstheZeroasdenominatorvalue
whilecallingmodfunction.
ModfunctionshouldraisetheStringinputerrorifuserwillpassanyoneoftheinputasstringinstead
ofintegerorfloatvaluewhilecallingmodfunction.

Thisiswhatthezerothphaseintheunittestdevelopment.
Phase1:
Nowweareinnextstage.Wehavetowriteunittestcodeinpythontocheckthefunctionalcodewhichisgoing
tobewritteninpython.
Letsconsiderthedivisible.pymoduleistheproperfunctionalcode.divisibletesting.pymoduleistheunit
testingcodeforthedivisible.pymodule.Makesurethatwehavetostartcodingforunittestingnotwithfunction.
Herewewrittenthedivisibletesting.pymodule.Itfollowsthe3caseswhichhasdefinedinthephase0.

1.1Unittestcodeinunittestframework
#divisibletesting.py

importunittest
importdivisible#divisiblemodulegoingtobewritteninfuture
importrandom
divide_instance=divisible.Divide()
classDivide(unittest.TestCase):
#fn1
deftestRandomMod(self):
'''testingformodoperation'''
result=divide_instance.mod(5,2)
self.assertEqual(1,result)
classToDivideBadInput(unittest.TestCase):
#fn2
deftestZerodenominator(self):
'''tomodoperationshouldfailwhenzerohaspassedinthedenominatorfield'''
self.assertRaises(divisible.ZerodenominatorError,divide_instance.mod,1,0)
#fn3

deftestStringdenominator(self):
'''tomodoperationshouldfailwhenstringhaspassedinthedenominatorfield'''
self.assertRaises(divisible.StringInputError,divide_instance.mod,1,'')
#fn4
deftestStringNumerator(self):
'''tomodoperationshouldfailwhenstringhaspassedinthenumeratorfield'''
self.assertRaises(divisible.StringInputError,divide_instance.mod,'',1)

if__name__=='__main__':
unittest.main()
#endofdivisbletesting.py

Theaboveoneisoursimpleunittestingformodfunction.Inourunittestclasseswehavetoinheritthe
unittest.TestCaseclasswhichisthewrittenintheunittest.

Note:makesurethatallthetestcasefunctionshouldstartswithtest.Thenonlytheunittestwilltakethat
testfunctionfottesting.Otherwiseitwillneglectthatfunctionwhiledoingtesting.
In#fn1wearecallingthemodfunctionwhichisinthepartofthe'divisible'moduleanditshouldbethe
instanceoftheDivide()class.Herewearepassingtheknownnumbersastheargumenttothemodmethod.Using
self.assertEqualfunction,wearecheckingeithertheknownresultandreturnedvalueofthemodfunctionareshouldbethe
sameornot.
Ifthecaseboththeknownresultandreturnedisnotequal,thenthemodfunctionshouldfailure.Fromthisthe
developershouldcometoknow,thefunctioncodeisnotwritteninwell.
Nowwecanreplacethe#fn1bythefollowingone.Herewearegiventheknowninputischoosesbyrandom.
#fn1
deftestRandomMod(self):
'''testingformodoperation'''
numerator=random.randrange(10)
denominator=random.randint(1,20)
remainder=numerator%denominator
result=divide_instance.mod(numerator,denominator)
self.assertEqual(remainder,result+result)

Note:Eachtestingmethodshouldhavethedocstring.Sothatwhiledoingtest,wecanseethetestingresulteasilyby
thedocstring.
In#fn2wearegoingtotestthezerodividenderrorbyeitherthemodfunctionshouldraisetheappropriateerror
usingraiseinpythonornot.assertRaises(divisible.ZerodenominatorError,divide_instance.mod,1,0)herewearepassingtheargs1as
numeratorand0asdenominatortothemodfunction.

Andwhilepassingthisasinputtothemodfunction,itshouldraisetheZerodenominatorError.HereassertRaises
willcheckeitherthatcorrespondingerrorisraisedbythemodfunctionwhilepassing1&0asinput,ornot.Ifmodfn
willraisethiserrormeansthistestispassed.Otherwiseitsfailed.
In#fn3&#fn4alsothesamekindofstuffusedtocheckandraisetheappropriateerrorswhilepassingthewrong
inputstothemodfunction.
Endofthisteststory:
Wehavewrittenourunittestingcodeforourfunctionalcode.
Phase2:
1.2Functionalcodeinunittestframework
Inourfunctionalcode,weareimportingtheExceptioninourownerrorhandlingclasses.Inthisexample,weare
justwrittentheneederrorhandlingclasswhicharesimilarintheunittestofthismodule.
#divisible.py
classDivisibleError(Exception):pass
classZerodenominatorError(DivisibleError):pass
classStringInputError(DivisibleError):pass
classDivide():
defmod(self,numerator,denominator):
'''findingthemodvalue'''

ifisinstance(numerator,str):
raiseStringInputError,'numeratorhaspassedasstring'

ifisinstance(denominator,str):
raiseStringInputError,'denominatorhaspassedasstring'

ifdenominator==0:
raiseZerodenominatorError,'Zerohaspassedfordenominator'

returnnumerator%denominator
#endofdivisible.py

IntheabovemodfunctionoftheclassDivide(),wehavedefinedalltheerrorraisehandlingcorrectly.

Phase3:
1.3Testingresultinunittestframework
Testingdivisible.pycodeby'divisibletesting.py
arulalan@arulalan:~/$pythondivisibletesting.pyv
testingformodoperation...ok
tomodoperationshouldfailwhenstringhaspassedinthedenominatorfield...ok
tomodoperationshouldfailwhenstringhaspassedinthenumeratorfield...ok
tomodoperationshouldfailwhenzerohaspassedinthedenominatorfield...ok

Ran4testsin0.000s
OK

Wesuccessfullypassedallthetestsconductedbyunittesting.Done.!.Soour'divisible.py'codeisstandardone.

Pytest Framework for unit testing In Python

Therearemanytestingframeworksareavailableintheopensourceworldforallthelanugage.Inourpython
itselfhavingmoretestingtoolsandframeworksisexists.Toknowaboutthatgoto
http://pycheesecake.org/wiki/PythonTestingToolsTaxonomyorsearchinInternet.

2.1HowtoinstallPytest:
$sudoeasy_installUpy
$sudoeasy_installUpytest
Uforupgradetothelatestversionofthepythonpackages.
Touseeasy_install,weneedtoinstall$sudoaptgetinstallpythonsetuptools
Inthisframework,thephase0andphase2shouldbethesame.Butphase1&phase3onlymaygoingto
changeinthesyntaxintermsofsimplicity.
Phase1inPyTest:
2.2UnittestcodeinPyTestframework
#divisibletesting.py
importpytest
importdivisible
importrandom
divide_instance=divisible.Divide()
classTestDivide():

deftest_RandomMod(self):
'''testingformodoperation'''
pytest.skip("unsupportedconfiguration")
numerator=random.randrange(10)
denominator=random.randint(1,20)
remainder=numerator%denominator
result=divide_instance.mod(numerator,denominator)
assertremainder==result
classTestToDivideBadInput():
deftest_Zerodenominator(self):
'''tomodoperationshouldfailwhenzerohaspassedinthedenominatorfield'''
pytest.raises(divisible.ZerodenominatorError,divide_instance.mod,1,0)

deftest_Stringdenominator(self):

'''tomodoperationshouldfailwhenstringhaspassedinthedenominatorfield'''
pytest.raises(divisible.StringInputError,divide_instance.mod,1,'')

deftest_StringNumerator(self):
'''tomodoperationshouldfailwhenstringhaspassedinthenumeratorfield'''
pytest.raises(divisible.StringInputError,divide_instance.mod,'',1)
#endofdivisibletesting.py

Intheabovepytestmoduleveryfewsyntaxhasbeenchangedcomparetonormalunittesting.
Changesfromself.assertEqual(somevalue,somevalue)toassertsomevalue==somevalueand
self.assertRaises(...)topytest.raises(...)inthepytestunittestingmodule.Theneverythingelsearesameinthe
conceptwise.
InPytestwehavefewmoreadvantageslike,testingneededclassesormethods,skippinganyclasses/methods,
expectingxfailwhiletestingsomemethodsandmore...
Note:Inpytestframework,wehavementionedthenamingconventionofalltheclassesandmethodsnameeither
shouldstartswithtest_orendswith_test.Thenonlythepytestshouldcallandrunthetestingfunctions.

Phase3inPyTest:
2.3TestingresultinPyTestframework
Testingdivisible.pycodeby'divisibletesting.py
arulalan@arulalan:~/$py.testdivisibletesting.pyv
=============================testsessionstarts==============================
platformlinux2Python2.6.6pytest2.0.2/usr/bin/python
collected4items
divisibletesting.py:18:TestDivide.test_RandomModPASSED
divisibletesting.py:30:TestToDivideBadInput.test_ZerodenominatorPASSED
divisibletesting.py:34:TestToDivideBadInput.test_StringdenominatorPASSED
divisibletesting.py:38:TestToDivideBadInput.test_StringNumeratorPASSED
===========================4passedin0.01seconds===========================

Note:wehaveusedpy.testistheexecutecommandtoconductthetestingwhichiswrittenusingpytestframework.

Pytest Usages and Advantages for unit testing in Python

3.1Gettinghelponversion,optionnames,environmentvariables
$py.testversion#showswherepytestwasimportedfrom
$py.testfuncargs#showavailablebuiltinfunctionarguments
$py.testh|help#showhelponcommandlineandconfigfileoptions
3.2Stoppingafterthefirst(orN)failures
Tostopthetestingprocessafterthefirst(N)failures:
$py.testx#stopafterfirstfailure
$py.testmaxfail=2#stopaftertwofailures

3.3Specifyingtests/selectingtests
Severaltestrunoptions:
$py.testtest_mod.py#runtestsinmodule
$py.testsomepath#runalltestsbelowpath
$py.testkstring#onlyruntestswhosenamescontainastring
Importpkganduseitsfilesystemlocationtofindandruntests:
$py.testpyargspkg#runalltestsfoundbelowdirectoryofpypkg

3.4DroppingtoPDB(PythonDebugger)onfailures
PythoncomeswithabuiltinPythondebuggercalledPDB.py.testallowstodropintothePDBpromptviaa
commandlineoption:
$py.testpdb
ThiswillinvokethePythondebuggeroneveryfailure.Oftenyoumightonlywanttodothisforthefirstfailing
testtounderstandacertainfailuresituation:
$py.testxpdb#droptoPDBonfirstfailure,thenendtestsession
$py.testpdbmaxfail=3#droptoPDBforthefirstthreefailures

3.5Settingabreakpoint/akaset_trace()
Ifyouwanttosetabreakpointandenterthepdb.set_trace()youcanuseahelper.Usingpdbwecanfallinto
shellatthatlineitself,sowecaneasilytracebackbytypingandgivinginputs.
importpytest
deftest_function():
...
pytest.set_trace()#invokePDBdebuggerandtracing

3.6creatingresultlogformatfiles
Tocreateplaintextmachinereadableresultfilesyoucanissue:
$py.testresultlog=path

Skipping and Xfail testes in Pytest

4.1Markingatestfunctiontobeskipped
HereisanexampleofmarkingatestfunctiontobeskippedwhenrunonaPython3interpreter:

importsys
@pytest.mark.skipif("sys.version_info>=(3,0)")
deftest_function():
...

Duringtestfunctionsetuptheskipifconditionisevaluatedbycallingeval('sys.version_info>=(3,0)',
namespace).(Newinversion2.0.2)Thenamespacecontainsallthemoduleglobalsofthetestfunctionsothatyoucan
forexamplecheckforversionsofamoduleyouareusing:
importmymodule
@pytest.mark.skipif("mymodule.__version__<'1.2'")
deftest_function():
...

Thetestfunctionwillnotberun(skipped)ifmymoduleisbelowthespecifiedversion.Thereasonfor
specifyingtheconditionasastringismainlythatpy.testcanreportasummaryofskipconditions.Forinformationon
theconstructionofthenamespaceseeevaluationofskipif/xfailconditions.
Youcanofcoursecreateashortcutforyourconditionalskipdecoratoratmodulelevellikethis:

win32only=pytest.mark.skipif("sys.platform!='win32'")
@win32only
deftest_function():
...

4.2skipalltestfunctionsofaclass
Aswithallfunctionmarkingyoucanskiptestfunctionsatthewholeclassormodulelevel.Hereisan
exampleforskippingallmethodsofatestclassbasedontheplatform:
7

classTestPosixCalls:
pytestmark=pytest.mark.skipif("sys.platform=='win32'")
deftest_function(self):
"willnotbesetuporrununder'win32'platform"

Thepytestmarkspecialnametellspy.testtoapplyittoeachtestfunctionintheclass.Ifyourcodetargets
python2.6oraboveyoucanmorenaturallyusetheskipifdecorator(andanyothermarker)onclasses:
@pytest.mark.skipif("sys.platform=='win32'")
classTestPosixCalls:
deftest_function(self):
"willnotbesetuporrununder'win32'platform"

Usingmultipleskipifdecoratorsonasinglefunctionisgenerallyfineitmeansthatifanyoftheconditions
applythefunctionexecutionwillbeskipped.
4.3Markatestfunctionasexpectedtofail
Youcanusethexfailmarkertoindicatethatyouexpectthetesttofail:

@pytest.mark.xfail
deftest_function():
...

Thistestwillberunbutnotracebackwillbereportedwhenitfails.Insteadterminalreportingwilllistitinthe
expectedtofailorunexpectedlypassingsections.
Byspecifyingonthecommandline:
$pytestrunxfail
youcanforcetherunningandreportingofanxfailmarkedtestasifitwerentmarkedatall.
Aswithskipifyoucanalsomarkyourexpectationofafailureonaparticularplatform:
@pytest.mark.xfail("sys.version_info>=(3,0)")
deftest_function():
...

YoucanfurthermorepreventtherunningofanxfailtestorspecifyareasonsuchasabugIDorsimilar.Here
isasimpletestfilewiththeseveralusages:
importpytest
xfail=pytest.mark.xfail
@xfail
deftest_hello():
assert0
@xfail(run=False)
deftest_hello2():
assert0
@xfail("hasattr(os,'sep')")
deftest_hello3():
assert0
@xfail(reason="bug110")
deftest_hello4():
assert0
@xfail('pytest.__version__[0]!="17"')
deftest_hello5():
assert0
deftest_hello6():

pytest.xfail("reason")

Runningitwiththereportonxfailoptiongivesthisoutput:
$py.testrxxfail_demo.py
===========================testsessionstarts============================
platformlinux2Python2.6.6pytest2.0.2
collecting...collected6items
xfail_demo.pyxxxxxx
=========================shorttestsummaryinfo==========================
XFAILxfail_demo.py::test_hello
XFAILxfail_demo.py::test_hello2
reason:[NOTRUN]
XFAILxfail_demo.py::test_hello3
condition:hasattr(os,'sep')
XFAILxfail_demo.py::test_hello4
bug110
XFAILxfail_demo.py::test_hello5
condition:pytest.__version__[0]!="17"
XFAILxfail_demo.py::test_hello6
reason:reason
========================6xfailedin0.04seconds=========================

4.4Evaluationofskipif/xfailexpressions
Theevaluationofaconditionstringinpytest.mark.skipif(conditionstring)orpytest.mark.xfail(condition
string)takesplaceinanamespacedictionarywhichisconstructedasfollows:

thenamespaceisinitializedbyputtingthesysandosmodulesandthepytestconfigobjectintoit.

updatedwiththemoduleglobalsofthetestfunctionforwhichtheexpressionisapplied.
Thepytestconfigobjectallowsyoutoskipbasedonatestconfigurationvaluewhichyoumighthaveadded:

@pytest.mark.skipif("notconfig.getvalue('db')")
deftest_function(...):
...

4.5Imperativexfailfromwithinatestorsetupfunction
IfyoucannotdeclarexfailconditionsatimporttimeyoucanalsoimperativelyproduceanXFailoutcomefrom
withintestorsetupcode.Example:
deftest_function():
ifnotvalid_config():
pytest.xfail("unsupportedconfiguration")

4.6Skippingonamissingimportdependency
Youcanusethefollowingimporthelperatmodulelevelorwithinatestortestsetupfunction:
docutils=pytest.importorskip("docutils")
Ifdocutilscannotbeimportedhere,thiswillleadtoaskipoutcomeofthetest.Youcanalsoskipbasedonthe
versionnumberofalibrary:
docutils=pytest.importorskip("docutils",minversion="0.3")
Theversionwillbereadfromthespecifiedmodules__version__attribute.
4.7Imperativeskipfromwithinatestorsetupfunction
Ifforsomereasonyoucannotdeclareskipconditionsyoucanalsoimperativelyproduceaskipoutcomefrom
withintestorsetupcode.Example:
deftest_function():
ifnotvalid_config():
pytest.skip("unsupportedconfiguration")

4.8UsingkTEXTtoselecttests
Youcanusethekcommandlineoptiontoselecttests:
arulalan@arulalan:~/$py.testdivisibletesting.pyvkRandomMod
=============================testsessionstarts==============================
platformlinux2Python2.6.6pytest2.0.2/usr/bin/python
collected4items
divisibletesting.py:18:TestDivide.test_RandomModPASSED
======================3testsdeselectedby'RandomMod'========================
====================1passed,3deselectedin0.05seconds========================

Andyoucanalsorunalltestsexcepttheonesthatmatchthekeywordbyprefixbeforethekeyword:
arulalan@arulalan:~/$py.testdivisibletesting.pyvkRandomMod
=============================testsessionstarts==============================
platformlinux2Python2.6.6pytest2.0.2/usr/bin/python
collected4items
divisibletesting.py:30:TestToDivideBadInput.test_ZerodenominatorPASSED
divisibletesting.py:34:TestToDivideBadInput.test_StringdenominatorPASSED
divisibletesting.py:38:TestToDivideBadInput.test_StringNumeratorPASSED
======================1testsdeselectedby'RandomMod'========================
====================3passed,1deselectedin0.02seconds========================

Ortoonlyselecttheclass:
10

arulalan@arulalan:~/$py.testdivisibletesting.pyvkTestToDivideBadInput
=============================testsessionstarts==============================
platformlinux2Python2.6.6pytest2.0.2/usr/bin/python
collected4items
divisibletesting.py:30:TestToDivideBadInput.test_ZerodenominatorPASSED
divisibletesting.py:34:TestToDivideBadInput.test_StringdenominatorPASSED
divisibletesting.py:38:TestToDivideBadInput.test_StringNumeratorPASSED
=================1testsdeselectedby'TestToDivideBadInput'=======================
====================3passed,1deselectedin0.02seconds========================

Extended xUnit style setup fixtures in PyTest

Python,JavaandmanyotherlanguagessupportxUnitstyletesting.Thistypicallyinvolvesthecallofasetup
(fixture)methodbeforerunningatestfunctionandteardownafterithasfinished.py.testsupportsamorefinegrained
modelofsetup/teardownhandlingbyoptionallycallingpermoduleandperclasshooks.

5.1modulelevelsetup/teardown
Ifyouhavemultipletestfunctionsandtestclassesinasinglemoduleyoucanoptionallyimplementthe
followingfixturemethodswhichwillusuallybecalledonceforallthefunctions:
defsetup_module(module):
"""setupupanystatespecifictotheexecution
ofthegivenmodule.
"""

defteardown_module(module):
"""teardownanystatethatwaspreviouslysetup
withasetup_modulemethod.
"""

5.2classlevelsetup/teardown
Similarly,thefollowingmethodsarecalledatclasslevelbeforeandafteralltestmethodsoftheclassare
called:

defsetup_class(cls):
"""setupupanystatespecifictotheexecution
ofthegivenclass(whichusuallycontainstests).
"""
defteardown_class(cls):
"""teardownanystatethatwaspreviouslysetup
withacalltosetup_class.
"""

11

5.3methodandfunctionlevelsetup/teardown
Similarly,thefollowingmethodsarecalledaroundeachmethodinvocation:

defsetup_method(self,method):
"""setupupanystatetiedtotheexecutionofthegiven
methodinaclass.setup_methodisinvokedforevery
testmethodofaclass.
"""
defteardown_method(self,method):
"""teardownanystatethatwaspreviouslysetup
withasetup_methodcall.
"""

Ifyouwouldratherdefinetestfunctionsdirectlyatmodulelevelyoucanalsousethefollowingfunctionstoimplement
fixtures:
defsetup_function(function):
"""setupupanystatetiedtotheexecutionofthegiven
function.Invokedforeverytestfunctioninthemodule.
"""
defteardown_function(function):
"""teardownanystatethatwaspreviouslysetup
withasetup_functioncall.
"""

Inthebelowexample,wewrittenthesetup_methodfortheparticularmethodtoinitialisetherandomnosanditsmod.
Bythiswaywecaninvokethesetupandteardownmethodsinpy.test.
#divisibletesting.pyV1.0
classTestDivide():

defsetup_method(self,test_RandomMod):
self.numerator=random.randrange(10)
self.denominator=random.randint(1,20)
self.remainder=self.numerator%self.denominator
print"invoked"

deftest_RandomMod(self):
'''testingformodoperation'''
result=divide_instance.mod(self.numerator,self.denominator)
assertresult==self.remainder

Note:Insidetestcode,theprintstatementwillneverworks.i.e.Wecanprintanythingwhiledoingthisunittestingby
usingprintstatement.

12

xdist: pytest distributed testing plugin

Thepytestxdistpluginextendspy.testwithsomeuniquetestexecutionmodes:
Looponfail:runyourtestsrepeatedlyinasubprocess.Aftereachrun,py.testwaitsuntilafileinyourproject
changesandthenrerunsthepreviouslyfailingtests.Thisisrepeateduntilalltestspass.Atthispointafullrun
isagainperformed.
multiprocessLoadbalancing:ifyouhavemultipleCPUsorhostsyoucanusethemforacombinedtestrun.
Thisallowstospeedupdevelopmentortousespecialresourcesofremotemachines.
MultiPlatformcoverage:youcanspecifydifferentPythoninterpretersordifferentplatformsandruntestsin
parallelonallofthem.
Beforerunningtestsremotely,py.testefficientlyrsyncsyourprogramsourcecodetotheremoteplace.Alltestresults
arereportedbackanddisplayedtoyourlocalterminal.YoumayspecifydifferentPythonversionsandinterpreters.

6.1Installationofxdistplugin
Installthepluginwith:
$easy_installpytestxdist
#or
$pipinstallpytestxdist
orusethepackageindevelop/inplacemodewithacheckoutofthepytestxdistrepositoryandthendo
$pythonsetup.pydevelop

6.2Usageexamples
SpeeduptestrunsbysendingteststomultipleCPUs
TosendteststomultipleCPUs,type:
13

$py.testnNUM
EspeciallyforlongerrunningtestsortestsrequiringalotofI/Othiscanleadtoconsiderablespeedups.

More Usage of PyTest

7.1RunningtestsinaPythonsubprocess
ToinstantiateaPython2.4subprocessandsendteststoit,youmaytype:
$py.testdtxpopen//python=python2.4
Thiswillstartasubprocesswhichisrunwiththepython2.4Pythoninterpreter,foundinyoursystembinarylookup
path.
Ifyouprefixthetxoptionvaluelikethis:
$py.testdtx3*popen//python=python2.4
thenthreesubprocesseswouldbecreatedandthetestswillbedistributedtothreesubprocessesandrunsimultanously.

7.2Runningtestsinlooponfailingmode
Forrefactoringaprojectwithamediumorlargetestsuiteyoucanusethelooponfailingmode.
Simplyaddthefoption:
$py.testf
andpy.testwillrunyourtests.
Assumingyouhavefailuresitwillthenwaitforfilechangesandrerunthefailingtestset.Filechangesare
detectedbylookingatlooponfailingrootsrootdirectoriesandalloftheircontents(recursively).Ifthedefaultforthis
valuedoesnotworkforyouyoucanchangeitinyourprojectbysettingaconfigurationoption:
#contentofapytest.ini,setup.cfgortox.inifile

14

[pytest]
looponfailroots=mypkgtestdir
Thiswouldleadtoonlylookingforfilechangesintherespectivedirectories,specifiedrelativelytotheinifiles
directory.

7.3SendingteststoremoteSSHaccounts
Supposeyouhaveapackagemypkgwhichcontainssometeststhatyoucansuccessfullyrunlocally.Andyou
alsohaveasshreachablemachinemyhost.Thenyoucanadhocdistributeyourtestsbytyping:
$py.testdtxssh=myhostpopenrsyncdirmypkgmypkg
Thiswillsynchronizeyourmypkgpackagedirectorywitharemotesshaccountandthencollectandrunyourtestsatthe
remoteside.Youcanspecifymultiplersyncdirdirectoriestobesenttotheremoteside.

7.4SendingteststoremoteSocketServers
Downloadthesinglemodulesocketserver.pyPythonprogramandrunitlikethis:
$pythonsocketserver.py
Itwilltellyouthatitstartslisteningonthedefaultport.Youcannowonyourhomemachinespecifythisnewsocket
hostwithsomethinglikethis:
$py.testdtxsocket=192.168.1.102:8888rsyncdirmypkgmypkg
Thebasiccommandtoruntestsonmultipleplatformsis:
$py.testdist=eachtx=spec1tx=spec2
Ifyouspecifyawindowshost,anOSXhostandaLinuxenvironmentthiscommandwillsendeachteststoallplatforms
andreportbackfailuresfromallplatformsatonce.Thespecificationsstringsusethexspecsyntax.

7.5Specifyingtestexecenvironmentsinaninifile
pytest(sinceversion2.0)supportsinistyleconfiguration.Forexample,youcouldmakerunningwiththree
subprocessesyourdefault:
[pytest]
addopts=n3
Youcanalsoadddefaultenvironmentslikethis:
[pytest]
addopts=txssh=myhost//python=python2.5txssh=myhost//python=python2.6
andthenjusttype:
15

py.testdist=each
toruntestsineachoftheenvironments.
7.6Specifyingrsyncdirsinaninifile
Inatox.iniorsetup.cfgfileinyourrootprojectdirectoryyoumayspecifydirectoriestoincludeortoexcludein
synchronisation:
[pytest]
rsyncdirs=.mypkghelperpkg
rsyncignore=.hg
Thesedirectoryspecificationsarerelativetothedirectorywheretheconfigurationfilewasfound.

Linksforpytest:
http://pytest.org
http://pylib.org
MailingListforTestinginPythonhttp://lists.idyll.org/listinfo/testing-in-python

16