Once MDL-43839 lands we may be interested in testing multiple window sizes, I will add a few watchers to comment about it.

I don't think that we need to run the whole suite using 3 different window sizes (eg: one single column, medium and big) but we might be interested in running specific scenarios in specific window sizes, and at the same time, to avoid running specific scenarios in some window sizes as tests may differ or only be applicable to some window sizes (hidden DOM parts, DOM nodes that moves over other DOM nodes when the window size changes...)

At the moment our suite runs and passes 100%; I say that because recently, working on MDL-44326, I've seen that a scenario was explicitly testing using a theme, something that we should avoid as 3rd parties may have a $CFG->theme hardcoded in config.php, allowing these scenarios they will have a problem because they will fail and they will not know if they have a problem with their local customizations or what; the same is applicable here with window sizes.

Looking for solutions, rather than creating a complex set of tags that users will need to know (and they don't want to) what I did was to skip the tests if "XXX" theme is not in use (https://github.com/dmonllao/moodle/compare/MDL-44326_master~6...MDL-44326_master~#diff-567c4381bb7a0c50e59c87deb5d89409R47) and if there are no better choices I think that we might be interested in doing the same when testing multiple window sizes; so, being practical, I mean that we can:

• Define a set of window sizes
• Set the window size when starting behat; otherwise we would need to add a step definition to allow scenarios to change it + reset it before each scenario (all scenarios needs to begin always from the same state: clean and without depending on the previous scenarios)
  • big size as always if nothing is specified
  • We could add a new behat option to do it without introducing users to the complex world of selenium capabilities; however, if we think that users will not be interested in running these scenarios we could simply manage it in our CI server or set our own (simple and not backend-browser-dependant... MDL-43932 hehehe) profiles. IMO the first option would be better.
• Add a step to the beginning of the scenarios that requires a specific window size, if the size specified when behat was started is not matching the required window size we skip the test

  Given a "(small|medium|large|whatever)" window size is being used