Modify the "Send Logs" feature

[misaochan]

The "send logs" feature in its current state (2.7.1) is not very useful. As mentioned at #1485 , there are two issues with it:

1. The buffer size seems to be very small, we get relatively few lines of logs compared to what Android Studio etc can store. We likely need to increase this.
2. We are getting a lot of unrelated logs from system etc. We need to filter our app's logs out if possible. @tanvidadu has suggested a method for this:

To filter out logcats https://stackoverflow.com/questions/17448912/using-grep-command-to-filter-logcat-in-android?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa this might help.

[maskaravivek]

I just came across https://github.com/hypertrack/hyperlog-android

Has anyone used it or have an opinion on it. It looks pretty cool and easy to setup.

[misaochan]

Copied from #1683 - In its current state, "send logs" is not very useful - the logs are not filtered for our app, and some logs that we receive don't contain anything from our app even after filtering. We need to investigate why this is happening and modify the feature (with filters) so that we can actually get useful logs from users.

[maskaravivek]

As I mentioned above, we could consider Hyperlog for this purpose. It will keep storing the logs in a local file and when the user clicks on send logs we can attach that file in the email and send it.

• It will give us the filtered logs for our application
• The number of logs to be stored can be controlled by us
• Its open source and looks to be quite easy to setup.

[nicolas-raoul]

https://security.stackexchange.com/a/190549/634 supports the idea of writing logs to a dedicated local file.
We could make the debug app write logs to both this file and logcat.

[maskaravivek]

IMO, for debug builds we already have logcat so writing to local file might not be very useful but for release build we could write just to local file. This file can then be attached while sending a mail. Am curious, isn't this what we already do?

[maskaravivek]

I will start working on this soon. It would be nice to gather opinions from others about what logging/crash monitoring framework we should be using.

Here's what I am considering:

• Hyperlog for storing logs and pushing them to a remote location
• Implementing our own file logging tree
• Use Fabric's crashlytics for error logging
• Use sentry for logging and error reporting

@dbrant It would be great if you could provide us with some insights on how we should go about it so that we do not violate any of the privacy policies.

[misaochan]

IMO, for debug builds we already have logcat so writing to local file might not be very useful but for release build we could write just to local file. This file can then be attached while sending a mail. Am curious, isn't this what we already do?

@maskaravivek Correct, I think, but for some reason none of the usual logs are showing up at all. You can take a look at the logs submitted at https://groups.google.com/forum/#!forum/commons-app-android-private to see (I believe you have access, right?), or test the "Send Logs" feature on the app and see.

For filtering and storage, hyperlog looks good to me but it would be best to wait for others to comment too.

[misaochan]

Basically, the scope of this task is:

• Figure out why the usual logs from our app are not being shown (specifically, in Release builds)
• Perform filtering for our app's logs (or use hyperlog if it automatically does that)
• Make sure storage is able to handle relatively large log file sizes (e.g. it should be able to store everything from the start of an upload to the end)

[nicolas-raoul]

Crashlytics would send logs to a third-party server, which is not acceptable for privacy reasons.
With sentry I guess we would need to set up a server, which is quite challenging too in terms of getting approval and maintaining security.
Hyperlog (locally used) sounds good :-)