oj@ jdZddlZddlZddlmZddlmZddlmZm Z m Z m Z m Z m Z mZmZmZmZddlmZmZddlmZmZmZddlmZdd lmZmZmZdd lm Z m!Z!dd l"m#Z#dd l$m%Z%dd l&m'Z'ddl(m)Z)ddl*m+Z+ddl,m-Z-ddl.m/Z/m0Z0ddl1m2Z2ddl3m4Z4ddl5m6Z6m7Z7ddl8m9Z9m:Z:ddl;mZ>m?Z?m@Z@mAZAddlBmCZCmDZDmEZEGdde e!ZFdeFfdZGeGdd ZHeGd!d"ZIeGd#d$ZJd%eKd&e/deKfd'ZLd(eId)e e0d&ez(StartedWorkerPool.run..s  4:M,v. / sN)rrS)selfrTs `r:runzStartedWorkerPool.runs6 >Boo     s "-+-c Kt|jDcgc]%}tj|j 'c}dd{}t |D]+\}\}}|r |j jd|d|-|jdd=|jdd=|jj|jjycc}w7w)zv Shut down all of the workers in the pool. The pool is unusable after this method is called. T) consumeErrorsNzjoining disttrial worker #z failed) rrRr fromCoroutineexit enumerate_loggerfailurerSrQcloserPunlock)r[rYresultsn succeededrcs r:joinzStartedWorkerPool.joins %AE NvX # #FKKM 2 N  (1'9 W #A# 7 $$'A!G%LgV W LLO OOA   ! O s#C$*C C$ C" C$(A;C$)r;N)r4r5r6r7rrrLrr r r/r0rrbr1r\rir8r9r:rOrOcsX$sm# O + ^$$hGl3&7D"r9rOc \eZdZUdZeed<deedee de de e fdZ dZd edefd Zy ) WorkerPoolz Manage a fixed-size collection of child processes which can run tests. @ivar _config: Configuration for the precise way in which the pool is run. _config protocolsrHrJr;c t|Dcgc]*\}}t||jt||,c}}Scc}}w)a Create local worker protocol instances and return them. @param protocols: The process/protocol adapters to use for the created workers. @param workingDirectory: The base path in which we should run the workers. @param logFile: The test log, for workers to write to. @return: A list of C{quantity} C{LocalWorker} instances. )rar/childrM)r[rmrHrJxprotocols r:_createLocalWorkerszWorkerPool._createLocalWorkerssF* )3 8 "2"8"8Q"@' J   s/Ac tdjj}ddddddtdtdi}t j j}t jjtj|d<|D]@}tj|g}|j|||tj|||By ) a Spawn processes from a list of process protocols. @param spawner: A C{IReactorProcess.spawnProcess} implementation. @param protocols: An iterable of C{ProcessProtocol} instances. @param arguments: Extra arguments passed to the processes. ztwisted.trial._dist.workertrialrwr'rr PYTHONPATH)argschildFDsenvN) rfilePathpathr(r)osenvironcopypathseprisys executableextend) r[spawnerrm argumentsworkertrialPathrxr}rYrws r:_launchWorkerProcessesz!WorkerPool._launchWorkerProcessess((IJSSXX s s s s   **//#!#  9  WFNNO4D KK " FCNNg V Wr9r?cDKt|jj\}}t|jjr t |jj}n%|j |jj}t|}t|jjDcgc] }t}}|j|||}|j|j||jjt|||||Scc}ww)z Launch all of the workers for this pool. @return: A started pool object that can run jobs using the workers. )r%rlrHrrJr preauthChildr&rangerGr0rrr spawnProcessrIrO) r[r?testDirrP testLogPathrQrprSrRs r:startzWorkerPool.starts 4 LL ) )  %% &"4<<#7#78K "..t||/C/CDKk*05dll6M6M0NO1n&O O**     ##   LL ( ( !        PsB3D 5DAD N)r4r5r6r7rFrLrr0rrr r r/rrrrrOrr8r9r:rkrksb  N+ #3-   k   0W6& ?& 7H& r9rk untilFailureresultc*|xr|jS)z Determine whether the test suite should be iterated again. @param untilFailure: C{True} if the suite is supposed to run until failure. @param result: The test result of the test suite iteration which just completed. ) wasSuccessful)rrs r:shouldContinuers  2F0022r9pool testCases driveWorkercK |jt|||d{y7#t$r0|jj t dt YywxYww)Nz )r\r ExceptionoriginaladdErrorr$r)rrrrs r:runTestsrsTFhhw{FI>??? F   L!979E Fs1A'+)+A'+6A$!A'#A$$A'ceZdZUdZeZeZede fe d<e e d<e e e d<dZee d<edee  Zee d <edeej, Zee d <d Ze e d<dZee d<dZee d<dZe e d<dZe e d<eZee gefe d<defdZ!dZ"dede#e$de%ddfdZ& d$de'e(e)fdedefdZ*d e'e(e)fdede fd!Z+d e'e(e)fde fd"Z,d e'e(e)fde fd#Z-y)%DistTrialRunnera A specialized runner for distributed trial. The runner launches a number of local worker processes which will run tests. @ivar _maxWorkers: the number of workers to be spawned. @ivar _exitFirst: ``True`` to stop the run as soon as a test case fails. ``False`` to run through the whole suite and report all of the results at the end. @ivar stream: stream which the reporter will use. @ivar _reporterFactory: the reporter class to be used. ._reporterFactory _maxWorkers_workerArgumentsF _exitFirstN)factory)default converter_reactorstreamr_tracebackFormat_realTimeErrors_uncleanWarningsztest.log_logfile _trial_temp_workingDirectory_workerPoolFactoryr;c|j|j|j|j}|jr t |}|j |S)zL Make reporter factory, and wrap it with a L{DistReporter}. )realtime)rrrrrr#_distReporterFactory)r[reporters r: _makeResultzDistTrialRunner._makeResultJsV(( KK..9M9M)   5h?H((22r9c$|jy)z Write test run final outcome to result. @param result: A C{TestResult} which will print errors and the summary. N)done)r[rs r: writeResultszDistTrialRunner.writeResultsUs  r9rrrYcNKfd}|D]}||d{y7w)a Drive a L{LocalWorkerAMP} instance, iterating the tests and calling C{run} for every one of them. @param worker: The L{LocalWorkerAMP} to drive. @param result: The global L{DistReporter} instance. @param testCases: The global list of tests to iterate. @return: A coroutine that completes after all of the tests have completed. cK j|d{y7#t$r'jj|t YywxYwwrW)r\rrrr)caserrYs r:taskz*DistTrialRunner._driveWorker..taskqsD :jjv... :((wy9 :s1A" "A"-AAAANr8)r[rrrYrrs ` ` r: _driveWorkerzDistTrialRunner._driveWorker]s-( :  Dt*    s %#%suiterc Ktt|jtt t j tjjj}jjd|jd|jjd{t!t"}t$dt&dt(ffd } t+||d{j-d{S7_77 #j-d{7wxYww)a* Spawn local worker processes and load tests. After that, run them. @param suite: A test or suite to be run. @param untilFailure: If C{True}, continue to run the tests until they fail. @return: A coroutine that completes with the test result. zRunning z tests. Nrgr;cKr"jjd|dzdjjrfd}nd}t t |j d{jS7w)Nz Test Pass r' c8jjSrW)rr)_rs r:z@DistTrialRunner.runAsync..runAndReport..s6??+H+H+Jr9cyNTr8)rs r:rz@DistTrialRunner.runAsync..runAndReport..sr9)rwriterrrr.rr)rgcasesConditionrr[ startedPoolrrs @r: runAndReportz.DistTrialRunner.runAsync..runAndReports !!Jq1ugR"89%%'F"K!/.)4!!      f %M sA/B 3B 4B )listr rrFminlenrrrrrrrcountTestCasesrrrrr+rKr*r-ri)r[rr poolStarter conditionrrrs` ` @@r:runAsynczDistTrialRunner.runAsynczs!"u-. -- C ND$4$45//0%%    HU%9%9%;$>""$ $ $M=F? %+""$ $ $sZCE D'2E ;D- D) D-E !D+"E )D-+E -EEEE testcBdddtttfddffd }dttt ffd }dt dt ffd }j jdd |t jj||j|j|j jttrjtts Jd ttj S) NFrur;c |yrWr8)rurs r:capturez%DistTrialRunner._run..capturesFr9c2djSyr)cancel)reactorStoppingrtestsInProgresssr:maybeStopTestsz,DistTrialRunner._run..maybeStopTestss""O~&&(&&r9rc@sjj|SrW)rstop)rrr[s r:maybeStopReactorz.DistTrialRunner._run..maybeStopReactors" ""$Mr9beforeshutdownz is not DistReporter)r rr*r robjectraddSystemEventTriggerr_raddBothr\ isinstanceraiseExceptionrr!r) r[rrrrrrrrs ` @@@r:_runzDistTrialRunner._runs59 % uWl23   &)9 :  V   ++Hj.Q  " "4==|#D E WW  W% &   fg &  ! ! # &,/PF8;O1PP/Iv//r9c(|j|dS)za Run a reactor and a test suite. @param test: The test or suite to run. Frrr[rs r:r\zDistTrialRunner.runs yyEy22r9c(|j|dS)z| Run the tests with local worker processes until they fail. @param test: The test or suite to run. Trrrs r:runUntilFailurezDistTrialRunner.runUntilFailures yyDy11r9)F).r4r5r6r7r*rrrbrr!rLrKr rMrboolrrrDrr3rstdoutrr rrrrrrkrrFrrr r"r0rr rrrrr\rr8r9r:rr!s (hGsI~..3iJ"'!/:#H 4?3::3NOFFO%c%!OT!"d"Hc*s*CM"2!3Z!?@M 3\ 3I&   @#J%Xy()J%J%  J%X*0x23*04*0I*0X3h 123y32E(I*=$>292r9r)Or7r|r functoolsros.pathrtypingrrrrr r r r r runittestrrattrsrrrattrs.convertersrtwisted.internet.deferrrrtwisted.internet.interfacesrrtwisted.loggerrtwisted.python.failurertwisted.python.filepathrtwisted.python.lockfilertwisted.python.modulesr _asyncrunnerr itrialr!r"rr#runnerr$utilr%r&r(r) distreporterr* functionalr+r,r-r.rYr/r0r1r3rDrFrOrkrrrrr8r9r:rsl     )'',HHE!*,20()543&MM== o E*E(*5"5"5"pb b b J 3 3y 3T 3F F "F F x *N;Yt_L F F$^2^2^2r9