Package Loading slow

[wackywendell]

I have 124 Atom packages in Atom, and linter-csslint is the slowest to load out of all of them: it takes 109ms to load, even when no project is opened and so no CSS files are in the workspace.

Could linter-csslint possibly do something like lazy loading?

[Arcanemagus]

Atom has activationHooks to solve this, unfortunately they are currently pretty buggy, a fix for this will be in the v1.4.0 branch (It's already in v1.4.0-beta0).

As soon as that gets pushed to stable I plan on moving all of the linter plugin packages to that model so they will only activate if you are loading a file of a language they support.

(Also you should think about switching to a different CSS linter like linter-stylelint, csslint itself hasn't seen any development for almost a year now and appears to be abandoned, I'm keeping this wrapper running for now but things currently don't look hopeful for csslint.)

[Arcanemagus]

Also lazy loading in the manner you linked wouldn't work as the package needs to register a service, not run a command itself.

There are probably a few things that could be done to speed up the current start up and I would happily review a PR for them but for the moment until v1.4.0 lands on stable I have no plans to implement this myself.

[dirk-thomas]

@wackywendell Can you please try the latest code in development mode since PR #78 should improve the situation and report back the latest status. Thanks.

[wackywendell]

Hmm... I just tried opening and closing Atom in development mode 10 times, first with 1.2.0, and then with master. I modified master slightly so that it reported a package number as 1.2.1, so that I could be sure it was loading master. With 1.2.0, I got an activation time of 40±2 ms, and with master, 55±15 ms. Maybe that difference is just noise... but I am not seeing much of an improvement. ☹ @dirk-thomas, were you seeing an improvement? Am I missing something?

[Arcanemagus]

A few things here: The time reported in timecop is the activation time while all other packages are also loading, so it will be pretty variable depending on when Atom decides to start loading it.

The slower activation time that #78 fixes only affects the very first package to call require('atom-package-deps').install() with no parameters (~30 ms time), subsequent calls are quite fast (~2 ms). So if you had a package calling that before this one did during startup you would see no difference from #78.

That's not to say that #78 isn't a good idea, just that it's not likely to show much of a difference unless this was the only package calling it in that manner 😉.

[dirk-thomas]

@Arcanemagus I don't think #78 only affect the activation time of the very first package. It also delays requiring the other dependencies (atom-linter, path).

@wackywendell For me the activation of this package without the patch (the hash before the patch was applied 31fbebc) is around 45 - 55 ms. With the patch applied it goes down to 30 - 35 ms. Both was measured in development mode (and calling npm install).

Both times are significantly faster if you only install the necessary dependencies and avoid installing the dev dependencies. So for non-development mode when installing the latest package version through atom (which doesn't install the dev dependencies) that would be the case. Still in this case #78 should improve the time by 10 - 15 ms.

@Arcanemagus Maybe you can provide some feedback why the pure existence of the dev dependencies in the node_modules folder impact the activation so negatively?

[Arcanemagus]

I have no idea why the devDependencies being present would affect the activation time 😮, @steelbrain?

I wonder if Atom is scanning all the folders in the dir for some reason? Recent versions of npm attempt to install the node_modules folder as "flat" as possible, while old versions (and thus apm install it hierarchically). If you do npm install --only=prod is the loading time the same as apm install?

Also I'd forgotten this was one of the PRs you also changed the require behavior, sorry 😉.

[steelbrain]

@Arcanemagus Having devDependencies present shouldn't affect activation time at all, not from my current understanding of the Node.js docs at least

[dirk-thomas]

For comparison I fetched the latest state of this package (33dbf16) and put it into the dev/packages folder. For the record this is with Atom 1.5.1. The activation time is very different depending on which node modules are installed:

• after npm install --only=prod it is either shown with 6ms or not listed (which implies 5ms or even faster)
  • npm list | wc -l output 16
• after npm install it takes 20 - 35 ms which is significantly slower!!!
  • npm list | wc -l output 271 just to give a rough comparison on the amount of installed node modules

Can anybody reproduce this? If yes, should this be reported somewhere else since I don't think how this would be related to linter-csslint itself?

[Arcanemagus]

Okay, I think I've discovered the difference @dirk-thomas: You are testing with atom --dev 😛

All testing was done on a blank profile, other than linter-csslint and linter (since it gets installed automatically). There was one more combination that could have been tested, which is linter-csslint installed from within the Atom package manager interface (equivalent to apm install --only=prod I'm guessing), but the spreadsheet was already large enough, and it wouldn't really affect the results in any major way 😉.

From the timing data from my tests you can see the following:

• At no point did linter-csslint's activation time pass the 5ms monitoring threshold
• Development packages being installed makes no difference unless Atom is launched with atom --dev, in which case linter-csslint's loading time is greatly increased, but activation time remains unchanged
• The hierarchical folder structure from apm install ("npm v2.13.3") is actually slower than npm (v3.3.12) flat structure, even though far more directories are in the root, it doesn't have to process as many levels to find the packages it needs.
• linter itself seems like it could use some work, as it took ~40 ms to activate

[dirk-thomas]

My measurements are showing the opposite of what you mention in the second bullet though. Starting atom -dev does increase the activation time for me. I would need to double check but I am pretty sure that load time was not affected at all.

Update: I read the second bullet wrong. So when using atom --dev it will load each an every node module (even if not used anywhere in the package) which will result in a much longer activation time. I thought this was a bug but its actually intended known behavior.

[Arcanemagus]

Oh, forgot to mention that I'm testing on Atom 1.6.0-beta2.

Bullet point 2 says nothing really about activation times, only package loading times. (I never once saw an activation time over 5ms, so I have no data on it...)

[dirk-thomas]

Even when you follow my exact steps to install linter-csslint in dev mode you don't see a higher activation time??

[Arcanemagus]

As stated I tested on a blank profile, with the only packages installed being linter-csslint and linter. (I'm not sure if this is a bug or not but atom --dev was loading all my normal packages as well, so I created a blank profile.)

Just to clarify, here is the equivalent to what I did:

mv ~/.atom ~/.atom.bak
cd ~/github/linter-csslint
apm link
npm install --only=prod
atom # to compile everything
atom /tmp/blank # check timecop, gather data, repeat 5 times
rm -Rf node_modules
npm install
atom /tmp/blank # check timecop, gather data, repeat 5 times
rm -Rf node_modules
apm install
atom /tmp/blank # check timecop, gather data, repeat 5 times