The bulk of Pygments is lexers for various languages. I think working on lexers is not a bad idea, as it will make for nicely self-contained projects. On the other hand, if it's a team of five and a project of eight weeks, just one such issue might not be enough. Luckily, there are dozens of them in the tracker ...

I think the project this team could tackle really depends on who they are. @ct456 What is your academic level? Did any of you contribute to open source software before? Is it your first year with programming or are you more advanced? Do you study computer science as a main subject or as part of a curriculum in another field? Do you have some experience of version control (whether Git, Bazaar, SVN, ...)?

Besides improving all the lexers, here are a few items I can think of, which are not necessarily captured in tracker issues (EDIT: maybe @Anteru and @birkenfeld have more to suggest):

Improving tests. Under scripts/, you can find a few tests that have not yet been made required, and would ideally be, fixing existing failures as much as possible or (if not feasible all at once) adding lists of known failures. This can seem less attractive because it doesn't directly impact users, but for development tests are paramount.
Adding new styles or improving existing formatters (the HTML one for example has some feature requests in the tracker).
Designing a more general mechanism for handling lists of builtins, perhaps with a JSON structure loaded from a file and passed as a parameter to the lexer, so that it will be easier to keep lists of builtins in sync with the evolution of each of the programming languages.
Improving regexlint, the tool we use to catch issues in regular expressions. My personal dream would be a check for catastrophic backtracking, which is a common issue affecting regexes (see the file CONTRIBUTING.md in the source) that can be easy to miss or give headaches to track. It would be very helpful to have a tool catching them automatically. I'd consider this a feasible project for CS students with a good background in algorithms and formal languages (but I have not experimented with it to see how hard it is).
For a group of master's students working full weeks: reimplement language recognition. Currently, lexers have an analyse_text method that returns a float between 0 and 1 indicating if the text is likely to be a program in the language in question. This isn't very reliable, see Refactor/Rewrite analyze_text #1702. There could be a mechanism based on AI trained with language data, on reusing the lexers' regexes, or a mix of both, to improve reliability. Of course this is a very ambitious target.

You will soon understand that while you will be working on this project for your university (at least at first, I hope you'll keep contributing afterwards :-), virtually everyone else is working on it for pleasure and fun. I recommend picking an issue that motivates you (of course, as long as it is also a reasonably sized goal for your allotted time and your current programming level). Also, feel free to ask any questions. I am myself a third-year math and CS student, so I've been in your shoes (first OSS contribution) just two years ago.

Please add a JSON5 lexer #1880

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions