Tokenizer/PHP: change tokenization of long PHP open tags #1015
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
c6f0061 to
f0ee904
Compare
a78b0dd to
c3fc316
Compare
547a345 to
93ce29b
Compare
Base automatically changed from
phpcs-4.0/feature/sq-1612-stdout-vs-stderr
to
4.x
April 17, 2025 13:09
f0ee904 to
f7b079f
Compare
|
Rebased without changes after the merge of #1010, which also gets rid of the merge conflict. |
... to no longer include whitespace (single space/new line).
This ensures the tokenization of all PHP open tags - long open tags `<?php`, short open tags `<?` and short open echo tags `<?=` - is consistent and should simplify handling of open tag tokens in sniffs and lower the cognitive load of sniff developers.
As per the detailed proposal in ticket 593.
Includes extensive unit tests, both ensuring the correct tokenization, as well as safeguarding consistency across the different open tags.
Includes the following additional updates to maintain the pre-existing behaviour:
* `AbstractPatternSniff`
The `AbstractPatternSniff` tokenizes an arbitrary code pattern to determine the token pattern to sniff for. This tokenization now to takes the change in the PHP open tag tokenization into account.
* `Generic/EmptyPHPStatement` - replace now redundant special condition for PHP open tags + update "fixed" file for improved output + add extra test.
* `PSR12/OpenTag` - allow for the changed tokenization.
The updated code also means that the fixer will no longer leave stray whitespace behind after a PHP open tag.
* `Squiz/CommentedOutCode` - allow for the changed tokenization.
* `Squiz/EmbeddedPhp` - code simplification.
* `FindStartOfStatementTest` - fixed token offset expectation as there is now an extra whitespace token.
* `ReplaceTabsInTokenMiscTest` - updated target finding
Fixes 593
f7b079f to
b00f2e4
Compare
|
Found a potential edge case bug while talking it through. Fixed now. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Change tokenization of long PHP open tags to no longer include whitespace (single space/new line).
This ensures the tokenization of all PHP open tags - long open tags
<?php, short open tags<?and short open echo tags<?=- is consistent and should simplify handling of open tag tokens in sniffs and lower the cognitive load of sniff developers.As per the detailed proposal in ticket #593.
Includes extensive unit tests, both ensuring the correct tokenization, as well as safeguarding consistency across the different open tags.
Includes the following additional updates to maintain the pre-existing behaviour:
AbstractPatternSniffTheAbstractPatternSnifftokenizes an arbitrary code pattern to determine the token pattern to sniff for. This tokenization now to takes the change in the PHP open tag tokenization into account.Generic/EmptyPHPStatement- replace now redundant special condition for PHP open tags + update "fixed" file for improved output + add extra test.PSR12/OpenTag- allow for the changed tokenization. The updated code also means that the fixer will no longer leave stray whitespace behind after a PHP open tag.Squiz/CommentedOutCode- allow for the changed tokenization.Squiz/EmbeddedPhp- code simplification.FindStartOfStatementTest- fixed token offset expectation as there is now an extra whitespace token.ReplaceTabsInTokenMiscTest- updated target findingSuggested changelog entry
Changed:
T_OPEN_TAGtokens will no longer contain any whitespace. [4.0 | Proposal: improve consistency for PHP open tag tokenization #593]T_WHITESPACEtoken.Related issues/external references
Fixes #593