Uploaded image for project: 'Moodle'
  1. Moodle
  2. MDL-65278

Random but usual behat failures with submission_types.feature

XMLWordPrintable

      Since some time ago we are detecting that the scenarios @ mod/workshop/tests/behat/submission_types.feature do fail randomly, but often than desired. That leads to lots of "yellows" and some "reds" in automated testing results. We should aim for more "greens" here.

      We have noticed that the problems happen often when CI workers are really busy and also under Oracle (which 2nd name is slowness). That may indicate that there is some point in the escenarios where behat is not waiting properly (by searching for something) or the standard js_pending()/js_complete() stuff is missing.

      Here there are the usual errors:

      Link: https://ci.moodle.org/job/S36.03.05%20-%20Behat%20-%20Chrome%20+%20Oracle/58/

      (you can see that 3 different failed and one of them failed twice, so the run ended red)

      Output:

      001 Scenario: Online text optional, file attachment required # /var/www/html/mod/workshop/tests/behat/submission_types.feature:110
            And I press "Start preparing your submission"          # /var/www/html/mod/workshop/tests/behat/submission_types.feature:118
              Button matching locator "'Start preparing your submission'" not found. (Behat\Mink\Exception\ElementNotFoundException)
       
      002 Scenario: File attachment only                   # /var/www/html/mod/workshop/tests/behat/submission_types.feature:150
            When I press "Start preparing your submission" # /var/www/html/mod/workshop/tests/behat/submission_types.feature:158
              Button matching locator "'Start preparing your submission'" not found. (Behat\Mink\Exception\ElementNotFoundException)
       
      003 Scenario: Neither submission type explicitly required # /var/www/html/mod/workshop/tests/behat/submission_types.feature:172
            And I press "Start preparing your submission"       # /var/www/html/mod/workshop/tests/behat/submission_types.feature:180
              Button matching locator "'Start preparing your submission'" not found. (Behat\Mink\Exception\ElementNotFoundException)
       
      420 scenarios (417 passed, 3 failed)
      9994 steps (9958 passed, 3 failed, 33 skipped)
      237m4.01s (107.13Mb)
      

      Ciao

            Unassigned Unassigned
            stronk7 Eloy Lafuente (stronk7)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved:

                Error rendering 'clockify-timesheets-time-tracking-reports:timer-sidebar'. Please contact your Jira administrators.