Upload UI tests #2626

domdomegg · 2019-03-17T00:56:52Z

Description

Fixes #705 (as I believe #2086 did unit tests, and this does UI tests so it should be all covered)

Increasing number of tests should decrease number of bugs
Key component of app is upload, this Espresso-based test validates it

Would hopefully catch issues like #2629

TODO

Having the test work automatically from a fresh install. Currently it requires running the app and logging in, and having a file called image.jpg in the base of the filesystem.

I'm not sure how to go about this, (hence the 'help needed'). I think this roughly breaks into two fairly separate parts:

Automatically logging in with credentials securely stored in Travis (not publicly on Github). Maybe this can be done in a similar way to how the APK signing keys work?
Updating the android image to have a sample image.jpg in /storage/emulated/0/ (symlink'ed from /mnt/sdcard and /sdcard/)

Also I'm worried that if the API is very slow, the delays I've put in the test at certain points to wait for responses won't be long enough - I'm not sure what we can do about this too.

These will need to be done to get this working on Travis, however as it's not actually running the Android tests at the moment we could merge anyways and fix this when we re-enable them.

Contributing

If you think you can help please do create a branch continuing mine and submit PRs back to domdomegg/test-upload :)

codecov-io · 2019-03-17T01:10:36Z

Codecov Report

Merging #2626 into master will decrease coverage by 0.01%.
The diff coverage is 0%.

@@            Coverage Diff            @@
##           master   #2626      +/-   ##
=========================================
- Coverage    2.77%   2.75%   -0.02%     
=========================================
  Files         259     260       +1     
  Lines       12365   12438      +73     
  Branches     1116    1125       +9     
=========================================
  Hits          343     343              
- Misses      11995   12068      +73     
  Partials       27      27

Impacted Files	Coverage Δ
...r/free/nrw/commons/contributions/MainActivity.java	`0% <0%> (ø)`	⬆️
app/src/main/java/fr/free/nrw/commons/Media.java	`1.53% <0%> (-0.04%)`	⬇️
...n/java/fr/free/nrw/commons/MediaDataExtractor.java	`0% <0%> (ø)`	⬆️
...rw/commons/mwapi/ApacheHttpClientMediaWikiApi.java	`0% <0%> (ø)`	⬆️
...fr/free/nrw/commons/settings/SettingsFragment.java	`0% <0%> (ø)`	⬆️
...ommons/contributions/ContributionsSyncAdapter.java	`0% <0%> (ø)`	⬆️
...java/fr/free/nrw/commons/upload/UploadService.java	`0% <0%> (ø)`	⬆️
...fr/free/nrw/commons/media/MediaDetailFragment.java	`0% <0%> (ø)`	⬆️
...nrw/commons/achievements/AchievementsActivity.java	`0% <0%> (ø)`	⬆️
...in/java/fr/free/nrw/commons/mwapi/CustomMwApi.java	`0% <0%> (ø)`	⬆️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d24bf20...fbceb9a. Read the comment docs.

maskaravivek · 2019-03-17T05:59:26Z

Great initiative @domdomegg! I can try helping with some of the things that you have mentioned in the help section.

logging in

It should be quite simple. We can probably get a test user created as discussed in #2226.

and having a file called image.jpg

Yes, this should be possible via travis.

Can you resolve merge conflicts so that i can work on top of it?

domdomegg · 2019-03-17T09:47:38Z

The idea in #2226 was to create a locked down user which we could share the password with so it could run instrumentation tests automatically on anyone's machine - for example tests on the contributions activity. I should have updated it before, but it doesn't seem possible as you can't block a user from changing their password. Also if a user is locked down it means we can't upload the test image.

Therefore I think we need a new test user which isn't locked down, but we just keep the login details secretly stored on Travis, and then expect people running tests requiring login on their own machines to login to their own test accounts.

…id-commons into domdomegg-test-upload

vanshikaarora · 2019-03-17T13:14:47Z

Hey @domdomegg , I want to contribute can you please tell about few test that you are not working upon so that I can write the unit tests :)

domdomegg · 2019-03-17T14:31:21Z

Hey @domdomegg , I want to contribute can you please tell about few test that you are not working upon so that I can write the unit tests :)

I think this PR requires quite a few Travis config changes which only a few people have access to. I've just submitted #2636 which is a WIP on WelcomeActivityTest (the tutorial) with some TODO ideas there I'm not currently working on.

vanshikaarora · 2019-03-17T15:55:17Z

@domdomegg As far I have noticed the previous tests were written in Kotlin, so from now onwards are we writing the unit tests in java or in kotlin?

domdomegg · 2019-03-17T16:02:55Z

@domdomegg As far I have noticed the previous tests were written in Kotlin, so from now onwards are we writing the unit tests in java or in kotlin?

Good point actually. All the unit tests are done in Kotlin, and the one instrumentation (android) test was done in Java so I continued in Java. Actually thinking about it properly I think we should use Kotlin for the instrumentation tests so all the tests are the same language.

…id-commons into domdomegg-test-upload

maskaravivek · 2019-03-18T05:33:46Z

Updating the android image to have a sample image.jpg in /storage/emulated/0/ (symlink'ed from /mnt/sdcard and /sdcard/)

@domdomegg I have raised a PR to fix this. Now in the setup phase, I generate an image programmatically and then use it for uploading.

https://github.com/domdomegg/apps-android-commons/pull/1

maskaravivek · 2019-03-18T05:35:03Z

Another thing that we need to take care of is:

Runtime Permissions. As of now, I had to grant permission beforehand.

Edit: have added GrantPermissionRule to handle runtime permissions.

maskaravivek · 2019-03-18T05:53:00Z

app/src/androidTest/java/fr/free/nrw/commons/UploadTest.java

+            e.printStackTrace();
+        }
+
+        String fileUrl = "https://commons.wikimedia.beta.wmflabs.org/wiki/File:" +


Can we enhance this bit as well? We can probably call media wiki api to assert if the image was uploaded.

Upload test

maskaravivek · 2019-03-18T07:59:24Z

Apart from this, I am working on automating the login bit. Do we have other action items for making this master ready?

domdomegg · 2019-03-18T08:00:14Z

Apart from this, I am working on automating the login bit. Do we have other action items for making this master ready?

I don't think so. There probably will be more annoying issues once we enable it in Travis though.

maskaravivek · 2019-03-18T08:11:18Z

I don't think so. There probably will be more annoying issues once we enable it in Travis though.

Yes, the final goal should be to have everything work on a fresh install itself.

misaochan · 2019-03-18T09:59:37Z

Wow, thanks @domdomegg ! :)

As the tests are run on Commons beta, I personally think it is OK to use a regular beta account. This:

Therefore I think we need a new test user which isn't locked down, but we just keep the login details secretly stored on Travis, and then expect people running tests requiring login on their own machines to login to their own test accounts.

Sounds like a good idea to me.

maskaravivek · 2019-03-18T15:49:48Z

@domdomegg Have raised https://github.com/domdomegg/apps-android-commons/pull/2 to automate the login process.

With automated login with test account

maskaravivek · 2019-03-18T19:21:09Z

@domdomegg Can we merge this PR now?

domdomegg · 2019-03-18T20:08:41Z

@domdomegg Can we merge this PR now?

I think so

domdomegg added 6 commits March 16, 2019 18:09

Fix SettingsActivityTest

f1366bd

Add test in SettingsActivity to set recent upload limit to 0

a078d44

Add tests to recent upload limit setting

a83ab17

Simplify activity test rule

b0a0fe7

Add UploadTest

af1dcf8

Log the URL where the file should be uploaded

8da3ae4

Merge branch 'master' into test-upload

0073b5d

Merge branch 'test-upload' of https://github.com/domdomegg/apps-andro…

0392666

…id-commons into domdomegg-test-upload

Merge branch 'master' into test-upload

d86daf1

domdomegg changed the title ~~[WIP] [Help needed] Upload UI tests~~ [WIP] Upload UI tests Mar 17, 2019

maskaravivek added 2 commits March 18, 2019 01:58

Merge branch 'test-upload' of https://github.com/domdomegg/apps-andro…

a9b7c92

…id-commons into domdomegg-test-upload

Generate an image file before testing uploads

58fb8c6

With runtime permissions

1f801be

maskaravivek reviewed Mar 18, 2019

View reviewed changes

maskaravivek and others added 2 commits March 18, 2019 11:51

With automatic login

f37405c

Merge pull request #1 from maskaravivek/uploadTest

e0738d4

Upload test

With automatic login

a1d8704

maskaravivek and others added 2 commits March 19, 2019 00:16

Get test credentials from travis

daaea6f

Merge pull request #2 from maskaravivek/uploadTest

fbceb9a

With automated login with test account

domdomegg changed the title ~~[WIP] Upload UI tests~~ Upload UI tests Mar 18, 2019

maskaravivek merged commit db5290e into commons-app:master Mar 18, 2019

domdomegg deleted the test-upload branch March 18, 2019 22:26

This was referenced Mar 18, 2019

Create unit tests for darkness-checking logic #2649

Closed

Convert instrumentation tests to Kotlin #2689

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upload UI tests #2626

Upload UI tests #2626

domdomegg commented Mar 17, 2019 •

edited

Loading

codecov-io commented Mar 17, 2019 •

edited

Loading

maskaravivek commented Mar 17, 2019

domdomegg commented Mar 17, 2019

vanshikaarora commented Mar 17, 2019

domdomegg commented Mar 17, 2019

vanshikaarora commented Mar 17, 2019

domdomegg commented Mar 17, 2019

maskaravivek commented Mar 18, 2019

maskaravivek commented Mar 18, 2019 •

edited

Loading

maskaravivek Mar 18, 2019

maskaravivek commented Mar 18, 2019

domdomegg commented Mar 18, 2019

maskaravivek commented Mar 18, 2019

misaochan commented Mar 18, 2019

maskaravivek commented Mar 18, 2019

maskaravivek commented Mar 18, 2019

domdomegg commented Mar 18, 2019

Upload UI tests #2626

Upload UI tests #2626

Conversation

domdomegg commented Mar 17, 2019 • edited Loading

codecov-io commented Mar 17, 2019 • edited Loading

Codecov Report

maskaravivek commented Mar 17, 2019

domdomegg commented Mar 17, 2019

vanshikaarora commented Mar 17, 2019

domdomegg commented Mar 17, 2019

vanshikaarora commented Mar 17, 2019

domdomegg commented Mar 17, 2019

maskaravivek commented Mar 18, 2019

maskaravivek commented Mar 18, 2019 • edited Loading

maskaravivek Mar 18, 2019

Choose a reason for hiding this comment

maskaravivek commented Mar 18, 2019

domdomegg commented Mar 18, 2019

maskaravivek commented Mar 18, 2019

misaochan commented Mar 18, 2019

maskaravivek commented Mar 18, 2019

maskaravivek commented Mar 18, 2019

domdomegg commented Mar 18, 2019

domdomegg commented Mar 17, 2019 •

edited

Loading

codecov-io commented Mar 17, 2019 •

edited

Loading

maskaravivek commented Mar 18, 2019 •

edited

Loading