Tests: remove code supporting jenkins and testswarm #2256
Conversation
There was a problem hiding this comment.
I'm wondering if it's worth doing it on this branch. The intention for now is to not land any new changes on 1-13-stable unless a severe security issue gets reported and we decide to patch it. In such a case, if I still have a working local TestSwarm setup (it works for me right now locally so why drop it) it may be simpler for me to run tests there.
It's still not completely automatic as I need to manually join swarm from all the supported browsers but it's better than nothing.
As we discussed, what might be better is to use the new runner for this purpose. However, we'd have to have a run that includes tests against all jQuery minors since 1.8 and against all supported browsers in 1.13 via BrowserStack. If we want to drop the TestSwarm code completely, I think I'd prefer to have a workflow with all these versions set up - even if I have to trigger it manually and then re-trigger all the versions that timed out., etc., a couple of times.
If you'd like to submit such a PR, I'd happily review it - and, if it works, I'd be fine with completely dropping TestSwarm code from 1-13-stable as well. Otherwise, it doesn't hurt to keep it here; it's already gone from main.
What do you think?
|
I'd still like to remove all the Testswarm code from this branch. If we're going to be testing on BrowserStack, I'd like to solely use the new test runner. I want to do that not only to remove our dependence on a library that receives minimal maintenance, but to dogfood the test runner and fix any issues that may come up. But, I see your point about needing a manually dispatchable workflow with all the right browsers. I'll set that up for both this branch and the main branch, which I know will have a different browser list. |
|
Cool! Let me know when the workflow is ready; if you don't mind, I'd prefer removing the TestSwarm code when we have it available. |
2bb5a5f to
5f8e8f5
Compare
|
@mgol I've added a workflow that can be manually dispatched that runs tests in the same browsers and all the same jQuery versions as swarm. Here is a sample run: https://github.com/timmywil/jquery-ui/actions/runs/9352420764 The IE11 failure using jQuery 1.8.3 is expected and also fails on Testswarm. There are a some flakey tests as well, but they often pass on retries. The test runner seems to be a little more forgiving with flakey tests than swarm. I did try running all supported jQuery versions in the same browser, but it was taking too long. 30min is the hard limit from BrowserStack. So, I split them up by major version instead. |
|
We discussed this during the team meeting. For posterity: the PR can land as soon as commits are modified to contain the |
…rowsers - reduce max worker restarts to avoid extra long runs Closes jquerygh-2256
Removes code from the 1-13-stable branch that was used to build on jenkins and run tests on testswarm.
This is assuming we can still test on browserstack locally using the new test runner.