Captions/depictions are not saved to Commons

[nicolas-raoul]

Steps to reproduce:

• Build https://github.com/commons-app/apps-android-commons/commits/structured-data branch

• Open app

• Upload picture

• Fill captions, for instance like this:
  

• Fill depictions, for instance like this:
  

• Go the web page showing the uploaded result: https://commons.wikimedia.org/wiki/File:Centre_de_transport_YAMATO_de_Meguro.jpeg

• See how it has neither the captions nor the depictions:

@macgills I believe you might want to assign this issue to yourself? Feel free to ask me any question :-)

[macgills]

Okay, yowza

@SuppressLint("CheckResult")
    private void uploadContribution(Contribution contribution) {
        Uri localUri = contribution.getLocalUri();
        if (localUri == null || localUri.getPath() == null) {
            Timber.d("localUri/path is null");
            return;
        }
        String notificationTag = localUri.toString();
        File localFile = new File(localUri.getPath());

        Timber.d("Before execution!");
        curNotification.setContentTitle(getString(R.string.upload_progress_notification_title_start, contribution.getDisplayTitle()))
                .setContentText(getResources().getQuantityString(R.plurals.uploads_pending_notification_indicator, toUpload, toUpload))
                .setTicker(getString(R.string.upload_progress_notification_title_in_progress, contribution.getDisplayTitle()));
        notificationManager
                .notify(notificationTag, NOTIFICATION_UPLOAD_IN_PROGRESS, curNotification.build());

        String filename = contribution.getFilename();

        NotificationUpdateProgressListener notificationUpdater = new NotificationUpdateProgressListener(notificationTag,
                getString(R.string.upload_progress_notification_title_in_progress, contribution.getDisplayTitle()),
                getString(R.string.upload_progress_notification_title_finishing, contribution.getDisplayTitle()),
                contribution
        );

        Observable.fromCallable(() -> "Temp_" + contribution.hashCode() + filename)
                .flatMap(stashFilename -> uploadClient
                        .uploadFileToStash(getApplicationContext(), stashFilename, localFile,
                                notificationUpdater))
                .subscribeOn(Schedulers.io())
                .observeOn(Schedulers.io())
                .doFinally(() -> {
                    if (filename != null) {
                        unfinishedUploads.remove(filename);
                    }
                    toUpload--;
                    if (toUpload == 0) {
                        // Sync modifications right after all uploads are processed
                        ContentResolver.requestSync(sessionManager.getCurrentAccount(), BuildConfig.MODIFICATION_AUTHORITY, new Bundle());
                        stopForeground(true);
                    }
                })
                .flatMap(uploadStash -> {
                    notificationManager.cancel(notificationTag, NOTIFICATION_UPLOAD_IN_PROGRESS);

                    Timber.d("Stash upload response 1 is %s", uploadStash.toString());

                    String resultStatus = uploadStash.getResult();
                    if (!resultStatus.equals("Success")) {
                        Timber.d("Contribution upload failed. Wikidata entity won't be edited");
                        showFailedNotification(contribution);
                        return Observable.never();
                    } else {
                        Timber.d("making sure of uniqueness of name: %s", filename);
                        String uniqueFilename = findUniqueFilename(filename);
                        unfinishedUploads.add(uniqueFilename);
                        return uploadClient.uploadFileFromStash(
                                getApplicationContext(),
                                contribution,
                                uniqueFilename,
                                uploadStash.getFilekey());
                    }
                })
                .subscribe(uploadResult -> {
                    Timber.d("Stash upload response 2 is %s", uploadResult.toString());

                    notificationManager.cancel(notificationTag, NOTIFICATION_UPLOAD_IN_PROGRESS);

                    String resultStatus = uploadResult.getResult();
                    if (!resultStatus.equals("Success")) {
                        Timber.d("Contribution upload failed. Wikidata entity won't be edited");
                        showFailedNotification(contribution);
                    } else {
                        String canonicalFilename = "File:" + uploadResult.getFilename();
                        Timber.d("Contribution upload success. Initiating Wikidata edit for entity id %s",
                                contribution.getWikiDataEntityId());
                        // to perform upload of depictions we pass on depiction entityId of the selected depictions to the wikidataEditService
                        if (contribution.getDepictionsEntityIds() != null) {
                            for (String s : contribution.getDepictionsEntityIds()) {
                                wikidataEditService.createClaimWithLogging(s, canonicalFilename);
                            }
                        }
                        wikidataEditService.createClaimWithLogging(contribution.getWikiDataEntityId(), canonicalFilename);
                        wikidataEditService.createLabelforWikidataEntity(contribution.getWikiDataEntityId(), canonicalFilename,
                                (Map) contribution.getCaptions());
                        contribution.setFilename(canonicalFilename);
                        contribution.setImageUrl(uploadResult.getImageinfo().getOriginalUrl());
                        contribution.setState(Contribution.STATE_COMPLETED);
                        contribution.setDateUploaded(CommonsDateUtil.getIso8601DateFormatShort()
                                .parse(uploadResult.getImageinfo().getTimestamp()));
                        compositeDisposable.add(contributionDao
                                .save(contribution)
                                .subscribeOn(ioThreadScheduler)
                                .observeOn(mainThreadScheduler)
                                .subscribe());
                    }
                }, throwable -> {
                    Timber.w(throwable, "Exception during upload");
                    notificationManager.cancel(notificationTag, NOTIFICATION_UPLOAD_IN_PROGRESS);
                    showFailedNotification(contribution);
                });
    }

I'mma guess this'll be the problem.

If you can give me the cliff notes of this process beyond make contribution object, send contribution object that'd help

[nicolas-raoul]

This happens even when you select a few depictions, if I understand correctly?
So it looks like the depictions are lost somewhere after selecting them... not sure how they are supposed to be carried around, but probably like the categories (which are correctly uploaded).
Sorry for not having much technical insight to bring to the investigation :-)

[macgills]

Thanks Nicolas.

@maskaravivek @misaochan any possibility of ye chiming in with detail on the component detail and data flow? I am working on enhancing the readability of UploadController right now

[misaochan]

IIRC @ashishkumar468 handled the overhaul of the upload process code last year, pinging him for input. My knowledge of the flow is somewhat outdated. :)

[macgills]

This is behaving completely absurdly.

By calling Single.just(contribution) the depictionEntityIds are being cleared on the Contribution. I have no logical explanation

[maskaravivek]

@macgills When exactly are the depictionEntityIds getting cleared?

On a related note, I just noticed that Parcelable isn't implemented for both captions and depictionsEntityIds. So if Contribution object is being passed around, we would be losing information.

[macgills]

@maskaravivek I saw that and thought "oh maybe when it gets sent to the service" but that still isn't the case because they communicate with messages not intents so no parcelling. There must be a shared reference somewhere, I'll try to find what is clearing it

[macgills]

I guess it is working now.... I'll open a PR but I still don't have an idea of quite just how many web requests we need to be performing