I would vote for this too. Actually I thought there was one already :|

As of 2 weeks back we improved the display of run results a lot. The html snapshot of the test results can now be retrieved separately, and the result pages are prettier.

Check:

• http://swarm.jquery.org/job/1
• http://swarm.jquery.org/result/10
• http://swarm.jquery.org/api.php?format=debug&action=result&item=10
• http://swarm.jquery.org/index.php?action=result&item=10&raw (html snapshot)

Not elaborate enough to fill up a continuous integration framework like Jenkins through JUnit, for that we need to aggregate individual tests and groups and get all that info. But depending on how the next few weeks play out we may not have to do that all from scratch. Right now TestSwarm is working nicely and triggers work great from Jenkins.

Unfortunately this is still not quite useful for a console usage, when user expects the output to reflect properly where exactly a failure has happened.

What is the chance of including something like this into the TestSwarm: 40fdef5?

I know this isn't useful for console usage. But that's a whole other story and boils down to a fundamental design choice that we need to take. In general I'm in favor of taking such a path, but please open another issue for tracking that. Right now TestSwarm is not built for console usage and as such does not provide that kind of interface.

The way 40fdef5 implements result_json looks generally OK. Although for it to work with Jenkins it needs a few fixes, and it needs to store more information (such as name and duration of each group of assertions, boolean outcome of each assertion, and textual details of each failure), and be less QUnit specific (e.g. storing the test object from QUnit's log() callback is not going to work).

But please create a pull request, it is definitely heading in the right direction.

I was trying to store as little information as possible (boiled it down to mere failures). But yes I agree it must be converted to a more widespread and usable JSON format. Let me extract that commit then (recent history got messy :)).

Here: #206

I made a quick wrap-up of my bunyip "rewrite" here: b8d0de4, and especially this TestSwarm API interface. Still quite raw, but the logic flow is clear I guess. Although it obviously bears no sense without detailed results from TestSwarm in json or any other format. Can we come up with some, 'cause otherwise it's useless? :|

@Krinkle,

...be less QUnit specific (e.g. storing the test object from QUnit's log() callback 
is not going to work).

Btw, while I absolutely agree with you on this, but a result logged by QUnit callback is quite generic already, for example, here's the log for successfully passed assertion:

{
  actual: "lang",
  expected: "lang",
  message: "Check object iteration",
  result: true
}

And here's for the failed one:

{
  actual: 6,
  expected: 7,
  message: "Looping over an array",
  result: false,
  source: "    at Object. (http://.../tests/Utils.html:69:2)"
}

I do not see anything QUnit specific here, or extra, do you guys? Seems quite generic to me. Or can you perhaps suggest any other more wide-spread format?

but a result logged by QUnit callback is quite generic already, for example, here's the log for successfully passed assertion:

@jayarjo Whether QUnit's log format is generic or not doesn't matter. What I meant is that one shouldn't include that directly, for many reasons. Just reference the relevant properties (even if that means all of them) into a new object.

QUnit's log format is as good a start as any. We can always improve it later.

Better format: #214