Save "raw" picture in history

[nicolas-raoul]

Problem: Smartphones increasingly use post-processing and AI to make picture more vivid/interesting for instance by exaggerating colours or blurring the background. Unfortunately that often makes the pictures less representative of reality. The original is lost.

Idea: Also store a "rawer" version of the picture, if the camera app lets us have it. That one could be saved first, and then the post-processed picture could be saved as a new version of it. All versions stay visible in the file history, making it easy for user to decide which version they want to use:

Note: The format and filenames of the "raw" versions (when they are saved) are very model-dependent and settings-dependent, but hopefully we can implement this for one model/setting as a proof of concept.

[deek121477]

Can I work on this issue?

[nicolas-raoul]

@deek121477 Yes thanks!
What phone ad camera app are you using? The first step would be to figure out how to get the "raw" picture, please post your findings here :-)

[deek121477]

Thank you for assigning me this issue @nicolas-raoul
The phone that I would be using is Android and Adobe Photoshop Camera
And I more thing as I am beginner to this project
Can this issue be labeled as a good first issue or not?

[nicolas-raoul]

It is a good first issue in the sense that it is limited to a rather specific area of the app.
It is more difficult than the average "good first issue", though, but I am sure you can do it :-)

On my Pixel 4 phone the default camera has a setting called RAW+JPEG control which saves a .dng file in the Raw folder. My Galaxy phone also has an option to save raw images as a .dng.

Unfortunately I just discovered that DNG file upload is not implemented yet: https://phabricator.wikimedia.org/T21153
If we have a good proof-of-concept I believe we could raise the priority of that Phabricator ticket.

So I suggest starting with "HDR originals" rather than RAW, since they use JPG rather than DNG.

1. Install the OpenCamera app and open it.
2. Tap the 3 dots, tap HDR.
3. Settings>Photo settings>Save all images for HDR mode.
4. Take a picture (it takes time as the app actually takes 3 pictures.
5. Go to Gallery, find the OpenCamera folder. It should have 4 pictures: Low brightness, normal brightness, high brightness, HDR result.

So for now the implementation would be:

• When uploading a picture, the Commons app first checks whether it is in a folder called "OpenCamera" and whether the 3 HDR pictures are present.
• If yes, upload all of the 4 pictures one after the other using the Commons "file history" feature shown in the screenshot above. The HDR result should be uploaded last.
  (The advantage is that in the future whenever more powerful HDR algorithms get developed, Commons people can regenerate the HDR images. Also, for pictures with a historical value, specialists will be able to see all details that are not visible in the result HDR.)

[deek121477]

Thank you so much for giving the guidance for approaching the issue.
I am done with OpenCamera Part and I have got 4 images too.

[deek121477]

While uploading images on the commons app I have uploaded only 1 image as there was no option for 4 images to be uploaded. Here I am attaching the link to the image that I have uploaded:

https://commons.m.wikimedia.org/wiki/File:Air_is_wild_with_the_leaves.jpg?markasread=44574435&markasreadwiki=commonswiki

[nicolas-raoul]

@deek121477 At https://commons.wikimedia.org/wiki/File:Air_is_wild_with_the_leaves.jpg you can see a link called Upload a new version of this file. You will have to discover what API calls the form behind that link use. :-)

[deek121477]

@deek121477 At https://commons.wikimedia.org/wiki/File:Air_is_wild_with_the_leaves.jpg you can see a link called Upload a new version of this file. You will have to discover what API calls the form behind that link use. :-)

Okay. I will try.

[deek121477]

While cloning git@github.com:deek121477/apps-android-commons.git

I am getting a message of: [ permission denied(public key)
Please make sure you have the correct access rights
and the repository exists.]
So can I use the git clone command of https://github.com/deek121477/apps-android-commons.git as in the android studio, all files are opening and working properly?

Also, is HAXM needed?

[nicolas-raoul]

Have you tried all troubleshooting steps listed at https://docs.github.com/en/authentication/troubleshooting-ssh/error-permission-denied-publickey ?

[deek121477]

Have you tried all troubleshooting steps listed at https://docs.github.com/en/authentication/troubleshooting-ssh/error-permission-denied-publickey ?

Not yet. I will try

[deek121477]

Have you tried all troubleshooting steps listed at https://docs.github.com/en/authentication/troubleshooting-ssh/error-permission-denied-publickey ?

Not yet. I will try

Can I use HTTPS instead of SSH?

[nicolas-raoul]

I think HTTPS would not allow you to make changes (I could be wrong).

You can start with HTTPS, though. Just make sure to make backups to not lose your changes.

[deek121477]

I think HTTPS would not allow you to make changes (I could be wrong).

You can start with HTTPS, though. Just make sure to make backups to not lose your changes.

Okay.I will continue with HTTPS, and make sure about my backups too.
I have gone through this article to understand difference between HTTPS and SSH.
https://ourtechroom.com/tech/https-vs-ssh-in-git/

[deek121477]

Actually, I tried making changes in the code
But it was a bit confusing to find out code where photos get uploaded in history

[deek121477]

Can you please help me?
Understanding the code base is taking much time.

[nicolas-raoul]

Hi @deek121477,
Have you found the chain of method calls that performs the actual file upload?
Would you mind posting the list of methods here?

[deek121477]

No.I haven't found the chain of method calls

[nicolas-raoul]

@deek121477 I would suggest running the app in debug mode from your IDE (such as Android Studio), setting breakpoints, and actually uploading a picture using the app.

[deek121477]

@nicolas-raoul
I tried running in debug mode. But it runs very slow on my laptop(4GB RAM). Also setting breakpoints was a bit confusing.
Is there any resource where I can learn more about setting breakpoints?

[sivaraam]

@deek121477 In case you're still looking for information about breakpoints, you could take a look at this resource: Breakpoints - Debug your app  |  Android Developers. You could also read the full guide here: Debug your app  |  Android Developers

[bhavanagarlapati]

Hi @nicolas-raoul Can I take this up?

[nicolas-raoul]

@deek121477 Any progress on this? :-)

@bhavanagarlapati Please take one issue at a time. Do not hesitate to come back to this one after sending a pull request for your currently assigned one. :-)

[deek121477]

@deek121477 Any progress on this? :-)

@bhavanagarlapati Please take one issue at a time. Do not hesitate to come back to this one after sending a pull request for your currently assigned one. :-)

No

[sivaraam]

@deek121477 Do you have plans to pursue this further? It's totally fine either way 🙂

[XxBrezxX]

is someone working on this? I wanna try this

[nicolas-raoul]

@XxBrezxX Please focus on one issue at a time :-) Thanks for your enthusiasm!

[ZIYANGSONG2003]

Has this issue been completely resolved? Can I try this question? I'm using an Apple phone and I don't know if I can try this problem.

[antungchen]

Not sure if i can try on this issue.

[nicolas-raoul]

@ZIYANGSONG2003 I don't think an Apple phone would help here sorry! :'-(

@antungchen It is yours!

[nicolas-raoul]

@antungchen Any progress to share? :-)

[sivaraam]

@antungchen Unassigning now due to lack of response. Feel free to ping us if you're still interested in looking into this issue.

[ChristoJobyAntony]

Hi @nicolas-raoul,
Was wondering if this issue active, I had a Pixel 7 which support storing images in RAW format which could be potentially useful to test this implementation.

[nicolas-raoul]

@ChristoJobyAntony Yes you can have it if you want. Thanks! :-)

[ChristoJobyAntony]

@nicolas-raoul I was able to set up the dev environment and play around the application. To simulate the situation, I had taken an image in RAW + JPEG format in my Pixel 7. The JPEG and the RAW file is saved in the same folder as PXL_<DATE>_<COUNT>. RAW-02.ORIGINAL.dng and PXL_<DATE>_<COUNT>. RAVW-01.COVER.jpg. However, since the application doesn't seem to accept DNG files, yet I am unable to select the RAW in the file picker of the application.
What would you suggest would be the best course of action to take at this point ?

[nicolas-raoul]

The existing app code lets the user select a JPG image. Once the user has chosen a JPG, your new code should:

1. Identify the matching RAW file.
2. Upload that RAW file to the Wikimedia Commons server.

[ChristoJobyAntony]

Hi @nicolas-raoul thank you for the breakdown on the implementation idea. I am currently trying to figure out the codebase to better understand the upload flow. I was able to dig the software design documentation to get a high level understanding of the flow. However, I am unable to identify the particular classes handling the upload logic, and I am unable to set up the debugger.

My current setup uses Android studio to build and run the application on my Pixel 7 in the :app:installBetaDebug variant.
I have tried debugging the application my attaching a listener, but this doesn't seem to be working, as I am not able to get any logs or stop execution on any breakpoints. Would you be able to suggest what might be amiss here ?

[ChristoJobyAntony]

Would appreciate any direction in this regard. I am quite stuck with the codebase and not able to find any concrete documentation to help me understand the application design.

[nicolas-raoul]

@ChristoJobyAntony When you upload an image, you will see some debug info in logcat. You can then find these log instructions in the code. :-)

[ChristoJobyAntony]

@nicolas-raoul Thank you for your suggestion, I was able to set up the debugger to break on one of the functions I suspected had to do with the Image being picked. From this, I sort of traced the call of functions that are called when a file has been picked. (FilePicker.java). Furthermore, I also was able to identify the class (UplodPresenter) that handles the metadata regarding the images before the image is uploaded. From my investigations, I have drafted a very rough plan of how to implement this feature and would appreciate your feedback on which approach would be better.

1. Add the raw file (if found) to the file lists as the image has been picked. This would mean that the user would observe two images in the selected menu.
2. Silently handle the addition of the Raw file on the within the Contribution class, and mark it as a "RAW" version of this image.

My understanding of the codebase is very minimal, I am sort of making a lot of assumptions here. Since I only have a couple of days to work on this project, I would appreciate your suggestion on what would be the most optimal pathway for this solution ?

[nicolas-raoul]

Approach 2 is the one to use. Thanks! :-)

You may want to experiment with uploading a new version of a file on the Wikimedia Commons beta server https://commons.wikimedia.beta.wmflabs.org/

[ChristoJobyAntony]

Thank you for the prompt response. As per your suggestion I tried to upload a new version (file) of the file to the beta server, my key takeaways were:

1. The website doesn't allow upload of any DNG files, and when I tried to upload it as a new version of the file, the website loaded for a really long time and crashed.
2. The new version being uploaded is using the old upload wizard (title=Special:Upload). There are parameters that specify that old filename (&wpDestFile=A_ginger_bread.jpg&wpForReUpload=1)
3. I believe that wpDestFile and wpForReUpload are being used to signify that a new version of file is being uploaded.
   
   Would you be able to confirm my findings ? I am now a bit worried that this would require a large amount of change to the codebase. Which make me worried if I will be able to tackle this issue in the given time frame. Also is there an API documentation for the wiki media's server ? I tried looking around a bit but couldn't find anything.

[nicolas-raoul]

@ChristoJobyAntony Thanks a lot for the research! Unfortunately it seems like Wikimedia (our upstream) has still not implemented DNG upload, please accept my apologies for missing that! It means it will unfortunately not be possible to implment this for now :'-(

Upstream ticket: https://phabricator.wikimedia.org/T21153

[ChristoJobyAntony]

@nicolas-raoul Oh, I see, thank you for the heads-up. I have been looking for a good first issue to work on for a very long time. I would greatly appreciate your suggestion on any issue I could work on which would help me get to know the repo a bit better.

Also, quite frankly, I need to contribute an OSS repo as a part of Uni assignment. And our team choose this repo since it was very well maintained and active. I would love to help this repo in any way possible. Thank you :-)

[nicolas-raoul]

@ChristoJobyAntony First this should be an easy one: #5855 , then you can try this more interesting one, still not too difficult but touching on database: #5808