Improvements to logging

[dhruvkb]

Problem

Currently the workflow scripts, notably those in push_data_to_ccos/ and sync_community_team/ are riddled with print() statements serving the function of logging. While this is a perfectly workable solution, it is by no means elegant. Also the process of manually indenting the messages in the print() statements is messy and error-prone and makes for hard to comprehend logs.

Alternatives

Python comes with a nifty logging module that should be used instead. Sprinkle some ANSI colouring and 🤯 .

Implementation

• I would be interested in implementing this feature.

[dhruvkb]

Just adding resources that might help anyone taking a shot at this issue.

• The Python library curses seems like a good solution to our indentation problem.
• Docker Compose wrote a class dedicated to handle log writes from parallel processes which involved writing to past lines.
• There are some really cool ANSI codes that, if supported, allow you to move your cursor on the window field.

[dhruvkb]

You can also use (and modify) the logging system added as a part of #72.

[avats-dev]

Taking up this one @dhruvkb! Might take some time for me to complete this.

[avats-dev]

Also, I'm unable to open issues in this repo. Can you explain why?

[dhruvkb]

@avats-dev creativecommons/.github#17 both explains and fixes the issue.

[avats-dev]

@dhruvkb Sorry, I was out of station for sometime, so I am starting now on this.

I'm thinking of using log and setting up a common log.py and utils.py that can be used by all the other scripts in repo. This would also refactor code and help resolving #60.
What do you think?

[dhruvkb]

Sounds good, but you need to ensure that all the scripts can run independently of each other.

[dhruvkb]

@avats-dev some new developments in this area that might be relevant:

• A better logs experience with GitHub Actions

[avats-dev]

Wow, that is good! thanks @dhruvkb for the update, I'll keep that in mind. 👍

[ghost]

@dhruvkb would colorlog be a good idea for this?

[dhruvkb]

@sukhdeepg it might be, it's hard to say without a demo. GitHub Actions now also supports colors for error and warning formatting so that might be a simpler and more native solution.

[ghost]

@dhruvkb understood 👍

[rajdesai24]

I am taking this @dhruvkb

[rajdesai24]

@dhruvkb I have finished up the logging part for push_data_to_ccos and I guess we can have a demo for that if all goes well we can implement it in all other places.

[dhruvkb]

@rajdesai24 please create a PR so that we can review it.

[rajdesai24]

@dhruvkb I have made a pull request for the issue, please review it and let me know if there is some blocker.

[rajdesai24]

@rajdesai24 please create a PR so that we can review it.

@dhruvkb I'll be creating a new pr with the resolved issues and proper logging everywhere please review it and do the necessary

[dravadhis]

@rajdesai24 are you working on this? If not can I take this up?

[rajdesai24]

Yes, it's done my pr is left for review but you can still talk to dhruvkb he'll guide you the best

[dravadhis]

No worries. I'll look for some other issues.

[rajdesai24]

@dhruvkb I will be opening a new PR with the same changes as my last PR got closed as it was a little off track with the repo as guided by @TimidRobot. Should I go ahead and make a new one?

[Cronus1007]

@rajdesai24 You can go ahead.

[rajdesai24]

Created new PR

[TimidRobot]

Fixed by #143