Skip to content

Use Qscintilla instead of implementing our own code editor #260

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Apr 23, 2015

Conversation

MKleusberg
Copy link
Member

This adds the QScintilla library to our project. I've changed the SqlTextEdit class to use the QScintilla editor instead of depending on a plain Qt widget and trying to improve it for code editing. This should build fine on Qt4 and Qt5 using either cmake or qmake - so everybody should have the chance to test this (and give feedback) :)

Most of the time the new code behaves just like the old one but a few things changed. Right now these features are missing:

  • No separate syntax highlighting colours for functions and tables
  • No auto completion for table fields

These features are new:

  • Better auto completion for keywords, functions and tables (icons, help texts, ...)
  • Shift+Tab works as expected
  • Brace matching
  • Auto indentation

The main thing however is that this saves us from adding more and more code for all sorts of things (search & replace, error highlighting, ...) which has already been written by other people.

@MKleusberg
Copy link
Member Author

As mentioned in #258 I don't see any license issues and especially none which we wouldn't have had anyway because of the plotting library...

@justinclift
Copy link
Member

Squash them into one commit? 😄

@MKleusberg
Copy link
Member Author

Yep, but because I'm still working on it I feel like it's better to wait because this way I can always go back to e.g. yesterday's version if something breaks. As soon as I'm happy with the changes I can out them into one commit 😉

@justinclift
Copy link
Member

Ahh yeah, makes sense. Didn't think of that. 😀

@justinclift
Copy link
Member

"... because of the plotting library."

Uh oh... ?

@MKleusberg
Copy link
Member Author

@justinclift
See https://github.com/sqlitebrowser/sqlitebrowser/blob/master/libs/qcustomplot-source/GPL.txt or http://www.qcustomplot.com/ for example - they are using the GPL as well. But as I said: it's an external library which we can always link dynamically if it should be required. Then it's like we're not touching it at all; if that was a problem the Apple Store would probably be empty 😉

@MKleusberg MKleusberg added the enhancement Feature requests. label Apr 15, 2015
@MKleusberg MKleusberg self-assigned this Apr 15, 2015
@justinclift
Copy link
Member

k, no worries then. 😄

@MKleusberg
Copy link
Member Author

Just did a bit of additional code cleanup, rebased the entire branch to the current master and fixed syntax highlighting in the Edit Table dialog -> Progress :)

@rp-
Copy link
Contributor

rp- commented Apr 15, 2015

Is it easy to strip the unused qscintilla sources? I guess we could spare all the python stuff. I see you already removed the qt3 support.

@MKleusberg
Copy link
Member Author

Yes, I've removed Qt3 code, documentation, examples, and the Qt Creator plugin. Now I've also removed the Python stuff (which wasn't used anywhere), the API definitions for Python code (not used anywhere), and all the lexers except for the SQL one. Removing the lexers required a bit more work, making it necessary to edit both build files as well as the libs/qscintilla/src/Catalogue.cpp file. But man, this saves us a lot of code in the repository! Thanks for drawing my attention to this :)

@MKleusberg
Copy link
Member Author

After removing the QScintilla files we don't need I merged all the commit so far into one to make the diff of this entire pull request much smaller.

Since then I've done a bit of restructuring in the code to make it easier to understand and compile faster. Then I've added the own syntax highlighting styles for SQLite function names and table names from the database and finally I've restored the auto completion feature for column names. It's so much simpler with QScintilla than it used to be with our original code 😄

So I think everything in this branch is working more or less as before now. So I think it's ready for testing now! If there are no remaining major bugs (like no building on some platforms, etc.) I would suggest merging this in a couple of days to avoid a lot of rabasing and have it tested by more people than when doing separate builds.

@justinclift
Copy link
Member

Compile fails on OSX:

clang++ -headerpad_max_install_names -stdlib=libc++ -mmacosx-version-min=10.7 -arch x86_64 -o sqlitebrowser.app/Contents/MacOS/sqlitebrowser release/main.o release/sqlitedb.o release/MainWindow.o release/CreateIndexDialog.o release/EditTableDialog.o release/PreferencesDialog.o release/AboutDialog.o release/EditDialog.o release/ExportCsvDialog.o release/ImportCsvDialog.o release/sqltextedit.o release/sqlitetypes.o release/csvparser.o release/ExtendedTableWidget.o release/Sqlite3Lexer.o release/Sqlite3Parser.o release/sqlitetablemodel.o release/FilterTableHeader.o release/SqlExecutionArea.o release/VacuumDialog.o release/DbStructureModel.o release/Application.o release/CipherDialog.o release/ExportSqlDialog.o release/SqlUiLexer.o release/moc_sqlitedb.o release/moc_MainWindow.o release/moc_CreateIndexDialog.o release/moc_AboutDialog.o release/moc_EditTableDialog.o release/moc_PreferencesDialog.o release/moc_EditDialog.o release/moc_ExportCsvDialog.o release/moc_ImportCsvDialog.o release/moc_sqltextedit.o release/moc_ExtendedTableWidget.o release/moc_sqlitetablemodel.o release/moc_FilterTableHeader.o release/moc_SqlExecutionArea.o release/moc_VacuumDialog.o release/moc_DbStructureModel.o release/moc_Application.o release/moc_CipherDialog.o release/moc_ExportSqlDialog.o release/moc_SqlUiLexer.o release/qrc_icons.o release/qrc_flags.o   -F/usr/local/Cellar/qt/4.8.6/lib -L/usr/local/Cellar/qt/4.8.6/lib -lsqlite3 -ldl -L/usr/local/lib -L/usr/local/opt/sqlite/lib -framework Carbon -L/Users/jc/git_repos/sqlitebrowser/src/../libs/qhexedit -L/Users/jc/git_repos/sqlitebrowser/src/../libs/antlr-2.7.7 -L/Users/jc/git_repos/sqlitebrowser/src/../libs/qcustomplot-source -L/Users/jc/git_repos/sqlitebrowser/src/../libs/qscintilla/Qt4Qt5 -lantlr -lqhexedit -lqcustomplot -lqscintilla2 -framework QtGui -L/usr/local/Cellar/qt/4.8.6/lib -F/usr/local/Cellar/qt/4.8.6/lib -framework QtCore -framework QtNetwork 
/usr/local/Cellar/qt/4.8.6/bin/moc -DSCINTILLA_QT -DSCI_LEXER -DQT_NO_DEBUG -DQT_GUI_LIB -DQT_CORE_LIB -DQT_SHARED -I/usr/local/Cellar/qt/4.8.6/mkspecs/unsupported/macx-clang-libc++ -I. -I/usr/local/Cellar/qt/4.8.6/lib/QtCore.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtCore.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtGui.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtGui.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/include -I. -I../include -I../lexlib -I../src -Irelease -F/usr/local/Cellar/qt/4.8.6/lib -D__APPLE__ -D__GNUC__ Qsci/qscilexer.h -o release/moc_qscilexer.cpp
/usr/local/Cellar/qt/4.8.6/bin/moc -DSCINTILLA_QT -DSCI_LEXER -DQT_NO_DEBUG -DQT_GUI_LIB -DQT_CORE_LIB -DQT_SHARED -I/usr/local/Cellar/qt/4.8.6/mkspecs/unsupported/macx-clang-libc++ -I. -I/usr/local/Cellar/qt/4.8.6/lib/QtCore.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtCore.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtGui.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtGui.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/include -I. -I../include -I../lexlib -I../src -Irelease -F/usr/local/Cellar/qt/4.8.6/lib -D__APPLE__ -D__GNUC__ Qsci/qscilexersql.h -o release/moc_qscilexersql.cpp
ld: library not found for -lqscintilla2
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [sqlitebrowser.app/Contents/MacOS/sqlitebrowser] Error 1
make[1]: *** [release] Error 2
make: *** [sub-src-make_default] Error 2
make: *** Waiting for unfinished jobs....

@MKleusberg
Copy link
Member Author

Weird, the snippet you pasted really looks odd to me because in the first line it looks like it's trying to link it all together to an application but in the two following lines it looks like it's still compiling the QScintilla library. The error message which must be thrown by the first line says that the library can't be found (yet). Not sure why it doesn't wait for it to be built... Can you maybe try again and post the entire log?

@justinclift
Copy link
Member

Will do. I was using -j9, which is probably responsible for the weird log. Gimme a few minutes though, as I'm doing something else atm. 😉

@justinclift
Copy link
Member

Just did a git pull from this branch and it compiled without error this time. *whew* 😉

Trying this new QtScintilla thing out, it seems pretty nice. The tab and shift-tab thing are working, and better code completion already:

qtscin1
qtscin2

@justinclift justinclift added this to the 3.6 milestone Apr 17, 2015
@justinclift
Copy link
Member

Note - I'm going to adjust the expected date for the next release to be one week from now instead of this weekend.

I definitely don't have time to do stuff for a release this weekend. Need to get an upgrade done for Gluster + Jenkins, now that Google is shutting off their OpenID 2.0 support. 😦

@MKleusberg
Copy link
Member Author

Cool! Glad to hear it's finally working :)
As far as I see this branch has all the features of master in it now, plus adding a few new ones like these:

  • Try typing 'substr(' (or any other function name) and it should show you a short explanation of the parameters and the function's purpose
    help
  • Code folding to allow folding of multi-line statements
    code_folding
  • Errors in the Execute SQL tabs are highlighted (sometimes because SQLite won't recognise completely invalid statements)
    error_highlighting

@justinclift
Copy link
Member

Regex?

@justinclift
Copy link
Member

Do changes in the font selector in the Preferences dialog correctly update stuff in the new widget?

I tested quickly and it didn't seem like it. 😉

@justinclift
Copy link
Member

qtscin3

@rp-
Copy link
Contributor

rp- commented Apr 19, 2015

works pretty good, I just have 2 small issues.

  1. Could we bring the auto completion popup only after 3? entered characters?
  2. It seems the sqlite datatypes are missing in the autocompletion list, maybe were already missing before.

@MKleusberg
Copy link
Member Author

The changes in the preferences dialog only affected the editor after restarting the application... This is fixed now, so you get an immediate effect! Thanks for reminding me of that :) The small arrow indeed indicates a drop down menu but you have to hold your mouse button down for some time to have it opened. A single click performs the default operation (Save As when not saved yet, and Save when already saved).

As @rp- suggested I've also changed the auto completion to start only after three characters and include some SQLite datatypes.

@justinclift
Copy link
Member

Does regex still work? (as per #215)

I'm trying to think of how to test it myself, but my brain is not clicking for it (focused on other stuff atm 😉).

@rp-
Copy link
Contributor

rp- commented Apr 19, 2015

what does REGEXP have todo with this?

@justinclift
Copy link
Member

Gah yeah... you're right. It's been a long day. 😉

@MKleusberg
Copy link
Member Author

Yes, this is a frontend-only change (or at least it should be), things like the regular expressions are not affected. So this means we should test the following:

  • Does it build well? (an entire library has been added...)
  • Typing, scrolling, Tab, etc. (i.e. basic text editor features)
  • Syntax highlighting
  • Auto completion (keywords, functions, tables, columns)
  • Other code editing features (brace matching, help texts, code folding...) [these are mostly new, so there's no comparison to the master branch possible here]
  • Accessing the contents correctly (the execute current line button and the execute selected SQL only feature!!)
  • Settings (colours, fonts, ...)
  • All this should work for all SQL editor widgets in the application. Besides the Execute SQL tabs, these are the SQL log views and the SQL preview in the Create/Edit Table dialog.

This is a first rough implementation of QScintilla support as SQL text
editor.

It should work mostly and build fine with qmake as well as cmake. The
new code supports all the features of the old one plus adding a few
subtle improvements. The main point of this, however, is reducing the
code we have to maintain and making it easier to add new features to the
editor.
If a syntax error is reported by SQLite during execution in the Execute
SQL tab of the main window try to locate the problematic SQL statement
and highlight it using a red squiggling line.
Only start auto completion after a three letter threshold.

Only take words for auto completion from the API word list, not from the
current document.
@MKleusberg
Copy link
Member Author

Just reorganised the pull request: The actual transition to the new widget is done in one commit now (the first one) while the other commits add new features or make improvements, so they can be viewed separately. I've also updated the QScintilla source code to the newly released version 2.9.

As I haven't heard of any objections I'll merge this now and see what happens :)

MKleusberg added a commit that referenced this pull request Apr 23, 2015
Use Qscintilla instead of implementing our own code editor
@MKleusberg MKleusberg merged commit 506804a into master Apr 23, 2015
@ghost
Copy link

ghost commented Apr 23, 2015

Really nice enhancement @MKleusberg!

@justinclift
Copy link
Member

Excellent. Lets get some people to test this. 😄

@justinclift
Copy link
Member

Hmmmm, just noticed an interesting behaviour with this. When compiling this on OSX single threaded (eg "make") it's compiling fine. But when compiling multi-threaded (eg "make -j9") it's sometimes barfing:

[sqlitebrowser (master)]$ qmake
[sqlitebrowser (master)]$ make -j9
[...]
clang++ -c -pipe -stdlib=libc++ -mmacosx-version-min=10.7 -O2 -arch x86_64 -fPIC -w -DSCINTILLA_QT -DSCI_LEXER -DQT_NO_DEBUG -DQT_GUI_LIB -DQT_CORE_LIB -DQT_SHARED -I/usr/local/Cellar/qt/4.8.6/mkspecs/unsupported/macx-clang-libc++ -I. -I/usr/local/Cellar/qt/4.8.6/lib/QtCore.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtCore.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtGui.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtGui.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/include -I. -I../include -I../lexlib -I../src -Irelease -F/usr/local/Cellar/qt/4.8.6/lib -o release/moc_SciClasses.o release/moc_SciClasses.cpp
clang++ -c -pipe -stdlib=libc++ -mmacosx-version-min=10.7 -O2 -arch x86_64 -fPIC -w -DSCINTILLA_QT -DSCI_LEXER -DQT_NO_DEBUG -DQT_GUI_LIB -DQT_CORE_LIB -DQT_SHARED -I/usr/local/Cellar/qt/4.8.6/mkspecs/unsupported/macx-clang-libc++ -I. -I/usr/local/Cellar/qt/4.8.6/lib/QtCore.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtCore.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtGui.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/lib/QtGui.framework/Versions/4/Headers -I/usr/local/Cellar/qt/4.8.6/include -I. -I../include -I../lexlib -I../src -Irelease -F/usr/local/Cellar/qt/4.8.6/lib -o release/moc_ScintillaQt.o release/moc_ScintillaQt.cpp
clang++ -headerpad_max_install_names -stdlib=libc++ -mmacosx-version-min=10.7 -arch x86_64 -o sqlitebrowser.app/Contents/MacOS/sqlitebrowser release/main.o release/sqlitedb.o release/MainWindow.o release/CreateIndexDialog.o release/EditTableDialog.o release/PreferencesDialog.o release/AboutDialog.o release/EditDialog.o release/ExportCsvDialog.o release/ImportCsvDialog.o release/sqltextedit.o release/sqlitetypes.o release/csvparser.o release/ExtendedTableWidget.o release/Sqlite3Lexer.o release/Sqlite3Parser.o release/sqlitetablemodel.o release/FilterTableHeader.o release/SqlExecutionArea.o release/VacuumDialog.o release/DbStructureModel.o release/Application.o release/CipherDialog.o release/ExportSqlDialog.o release/SqlUiLexer.o release/moc_sqlitedb.o release/moc_MainWindow.o release/moc_CreateIndexDialog.o release/moc_AboutDialog.o release/moc_EditTableDialog.o release/moc_PreferencesDialog.o release/moc_EditDialog.o release/moc_ExportCsvDialog.o release/moc_ImportCsvDialog.o release/moc_sqltextedit.o release/moc_ExtendedTableWidget.o release/moc_sqlitetablemodel.o release/moc_FilterTableHeader.o release/moc_SqlExecutionArea.o release/moc_VacuumDialog.o release/moc_DbStructureModel.o release/moc_Application.o release/moc_CipherDialog.o release/moc_ExportSqlDialog.o release/moc_SqlUiLexer.o release/qrc_icons.o release/qrc_flags.o   -F/usr/local/Cellar/qt/4.8.6/lib -L/usr/local/Cellar/qt/4.8.6/lib -lsqlite3 -ldl -L/usr/local/lib -L/usr/local/opt/sqlite/lib -framework Carbon -L/Users/jc/git_repos/sqlitebrowser/src/../libs/qhexedit -L/Users/jc/git_repos/sqlitebrowser/src/../libs/antlr-2.7.7 -L/Users/jc/git_repos/sqlitebrowser/src/../libs/qcustomplot-source -L/Users/jc/git_repos/sqlitebrowser/src/../libs/qscintilla/Qt4Qt5 -lantlr -lqhexedit -lqcustomplot -lqscintilla2 -framework QtGui -L/usr/local/Cellar/qt/4.8.6/lib -F/usr/local/Cellar/qt/4.8.6/lib -framework QtCore -framework QtNetwork 
ld: library not found for -lqscintilla2
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [sqlitebrowser.app/Contents/MacOS/sqlitebrowser] Error 1
make[1]: *** [release] Error 2
make: *** [sub-src-make_default] Error 2
make: *** Waiting for unfinished jobs....
rm -f libqscintilla2.a
ar cq libqscintilla2.a release/qsciscintilla.o release/qsciscintillabase.o release/qsciabstractapis.o release/qsciapis.o release/qscicommand.o release/qscicommandset.o release/qscidocument.o release/qscilexer.o release/qscilexersql.o release/qscimacro.o release/qsciprinter.o release/qscistyle.o release/qscistyledtext.o release/MacPasteboardMime.o release/InputMethod.o release/SciClasses.o release/ListBoxQt.o release/PlatQt.o release/ScintillaQt.o release/LexSQL.o release/Accessor.o release/CharacterCategory.o release/CharacterSet.o release/LexerBase.o release/LexerModule.o release/LexerNoExceptions.o release/LexerSimple.o release/PropSetSimple.o release/StyleContext.o release/WordList.o release/AutoComplete.o release/CallTip.o release/CaseConvert.o release/CaseFolder.o release/Catalogue.o release/CellBuffer.o release/CharClassify.o release/ContractionState.o release/Decoration.o release/Document.o release/Editor.o release/EditModel.o release/EditView.o release/ExternalLexer.o release/Indicator.o release/KeyMap.o release/LineMarker.o release/MarginView.o release/PerLine.o release/PositionCache.o release/RESearch.o release/RunStyles.o release/ScintillaBase.o release/Selection.o release/Style.o release/UniConversion.o release/ViewStyle.o release/XPM.o release/moc_qsciscintilla.o release/moc_qsciscintillabase.o release/moc_qsciabstractapis.o release/moc_qsciapis.o release/moc_qscilexer.o release/moc_qscilexersql.o release/moc_qscimacro.o release/moc_SciClasses.o release/moc_ScintillaQt.o
ranlib -s libqscintilla2.a
[sqlitebrowser (master)]$

Any ideas what could cause that?

@justinclift
Copy link
Member

On my system -j9 seems to reliably trigger the failure. -j5 and -j7 don't. Weird.

@MKleusberg MKleusberg deleted the qscintilla branch April 27, 2015 09:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Feature requests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants