Skip to content

Language Support

Jump to bottom
DennisKehrig edited this page Apr 10, 2013 · 95 revisions

Originally, most support for programming languages was implemented in core Brackets. Starting with Sprint 21, the language manager is used for adding basic language support, like syntax highlighting and comment toggling (for a list of supported languages, see languages.json). The language manager can also be used by extensions.

This page documents how language support in Brackets is currently implemented. The companion page [Language Support Changes](Language Support Changes) documents what additional capabilities extensions would need in order to add language support on par with the current level of support for HTML, JavaScript and CSS.

Our goal is to base all language-related features like code hinting, text manipulation commands, quick edit, live preview, etc. on these capabilities. Support for non-web programming languages will be moved moved to dedicated extensions (see issue #2969).

Retrieving a Language object

Each supported language is represented by exactly one Language object that can be retrieved in various ways:

  • When defining a language: LanguageManager.defineManager(id, definition).done(function (language) { ... });
  • Using a language ID: var language = LanguageManager.getLanguage("javascript");
  • Using a file path: var language = LanguageManager.getLanguageForPath("/path/to/file.js");
  • Using a Document instance: var language = doc.getLanguage();
  • Using an Editor instance*: var language = editor.getLanguageForSelection();
  • Using another Language instance and a CodeMirror mode: var language = otherLanguage.getLanguageForMode("xml");

*The language in a selection may not be the same as the language for the document. For example, the html language supports css and javascript content.

Except for LanguageManager.getLanguage, these methods always return a language object. To determine whether this is the fallback language, use if (language.getId() === "unknown") { ... }.

Using a Language object

A Language object contains model data for a language. The following methods are available:

  • getId() returns the ID of a language (e.g. "cpp", "cs"). Use this to trigger language-specific behavior.
  • getName() returns the human-readable name of a language (e.g. "C++", "C#"). Used by the status bar.
  • getMode() returns the CodeMirror mode for a language. Used by EditorManager, should only be used in combination with CodeMirror specific code. Use getId() to identify a language.
  • getFileExtensions() returns an array of file extensions for a language (e.g. "svg", "html.erb").
  • getFileNames() returns an array of file names for a language (e.g. "Makefile", ".profile").
  • hasLineCommentSyntax()/getLineCommentSyntax()/setLineCommentSyntax(prefix) returns line comment info. Used by the toggle line comment editor command.
  • hasBlockCommentSyntax()/getBlockCommentPrefix()/getBlockCommentSuffix()/setLineCommentSyntax(prefix) returns block comment info. Used by the toggle block comment editor command.
  • getLanguageForMode() returns either a language associated with the mode or the fallback language. Used to disambiguate modes used by multiple languages.

Defining a new language

In an extension, if a language has an existing CodeMirror mode, you can declare the new language in a simple JSON object:

var LanguageManager = brackets.getModule("language/LanguageManager");

LanguageManager.defineLanguage("haskell", {
    name: "Haskell",
    mode: "haskell",
    fileExtensions: ["hs"],
    blockComment: ["{-", "-}"],
    lineComment: ["--"]
});

If you need to provide a custom mode, it must be registered to CodeMirror using CodeMirror.defineMode() first before calling LanguageManager.defineLanguage().

Refining an existing language

Retrieve a Language object by its ID, then call setter methods to change it.

var LanguageManager = brackets.getModule("language/LanguageManager");

var language = LanguageManager.getLanguage("coffeescript");

language.addFileExtension("cf");
language.addFileName("Cakefile");
language.setLineCommentSyntax(["#"]);
language.setBlockCommentSyntax("###", "###");

For further details, please refer to the comments in LanguageManager.js.

Places contributing to "language" support as of Sprint 21

Based on LESS Refactoring. The remainder of this page documents future work required to support languages across various features in Brackets. The focus is on finding out what is hardcoded and needs to be refactored to use existing capabilities, or requires new capabilities before it can be refactored.

No changes required

These are okay the way the are.

  • document/DocumentManager.js
    • Document.getLanguage uses the LanguageManager to determine the language based on the file extension.
  • language/CSSUtils.js
    • Method extractAllSelectors extracts CSS selectors from a string. Internally uses CodeMirror's css mode as a parser, but that could be swapped out.
  • language/JSLintUtils.js (now extensions/default/JSLint/main.js)
    • Uses JSLint internally which could be swapped out.
  • language/JSUtils.js
    • Method findAllMatchingFunctionsInText to find all instances of a function name in a string of code. Internally uses CodeMirror's javascript mode as a parser, but that could be swapped out.
  • language/LanguageManager.js
    • Defines the Language class used to represent a given language
    • Loads default languages from language/languages.json
    • Method defineLanguage(id, definition) to define and register a language. Returns a promise object that will be resolved with the Language object.
    • Method getLanguage(id) resolves a language ID to a Language object.
    • Method getLanguageForPath(path) resolves a file path to a Language object.
    • Used by extension "LESSSupport" to add basic support for LESS
  • utils/StringUtils.js
    • Method htmlEscape escapes characters with special meaning in HTML. However, this function is necessary since the Brackets UI is written in HTML, and has nothing to do with language support for the users.

Straightforward refactoring

These need to be changed to use existing functionality.

  • Done: brackets.js requires language/JSLintUtils.js. This can be refactored into an extension without introducing new APIs. See issue #3094 and pull request #3143.
  • brackets.js requires editor/CSSInlineEditor.js. It should first call require("editor/MultiRangeInlineEditor") (loaded by CSSInlineEditor.js), since this defines shortcuts for inline editor navigation. Then the CSSInlineEditor could be moved to an extension.
  • editor/CodeHintManager.js
    • In progress: Method registerHintProvider registers hint providers by mode. This can simply be changed to check for language IDs since currently all modes this function is being called with (either by Brackets or the known extensions) belong to a language with an equal ID ("css", "html", "javascript"). See issue #3085 and pull request #3270.
  • Done: editor/CSSInlineEditor.js
    • Inline editor provider htmlToCSSProvider decides to open based on the editor mode. This can simply be changed to check for the language ID to be "html" (via editor.getLanguageForSelection().getId()).
  • project/FileIndexManager.js. See pull request #3301.
    • Maintains an index called "css" using only files ending with ".css", i.e. uses file extensions. The call to add this index should be moved to CSSUtils (the only place this index is used at the moment). In addition, the filter function can be changed to use the language API: return LanguageManager.getLanguageForPath(entry.name).getId() === "css";
  • Done: language/JSLintUtils.js
    • Method run to run JSLint on the current document. Checks if the extension is .js, therefore uses file extensions. See issue #3094 and pull request #3143.
  • language/JSUtils.js
    • Method findMatchingFunctions finds all functions with a specified name within a set of files. Filters these files by checking that the file extension is ".js", i.e. uses file extensions. This should use the language API instead (determine the language for the file and check whether that language has the ID "javascript").
  • Done: search/QuickOpen
    • Method addQuickOpenPlugin uses file extensions to register plugins. It should use language IDs instead. It is currently only used with file extensions "css", "js" and "html". For "css" and "html", the calling code can remain unchanged, transparently changing the meaning of the string from file extension to language ID. For "js", the calling code needs to use "javascript" instead. Currently extensions/default/QuickOpenJavaScript/main.js is the only place in either Brackets core or the extensions that uses this file extension. See pull request #3301.

Issues that should be adressed as part of other planned work

These are places that affect areas we already have plans to work on, and where issues are best adressed as part of that work.

  • document/DocumentCommandHandlers.js
    • Method _handleNewItemInProject hardcodes ".js"/"Untitled.js" as the default file extension/name for new files. Changing this to work as proposed in card #291 would remove this issue.
  • language/{CSSUtils|HTMLUtils|JSUtils}.js should be provided by default extensions. For this to work, all other parts that depend on them (ideally only other extensions) need to be able to access these extensions. Supporting this is part of the ongoing extensions research.
    • brackets.js loads JSUtils.js. This is only necessary so extensions can load it synchronously via JSUtils = brackets.getModule("language/JSUtils") instead of asynchronously via brackets.getModule(["language/JSUtils"], function (JSUtils) { ... }). This can be removed once JSUtils can be loaded as an extension.
    • brackets.js exports CSSUtils and JSUtils for tests. Tests should instead load these modules from extensions, but this depends on the point above.
    • editor/CSSInlineEditor.js relies on HTMLUtils and CSSUtils.
    • LiveDevelopment/Agents/DOMHelpers.js contains multiple methods that encapsulate knowledge about HTML, should potentially be moved to HTMLUtils
    • LiveDevelopment/Agents/DOMNode.js contains DOMNode.prototype.toString contains basic knowledge about HTML, should potentially be HTMLUtils
  • utils/ExtensionUtils
    • Method loadStyleSheet uses file extensions to support LESS files. Once we have a compiler infrastructure in place, any path could be mapped to a language, and if there's a compiler to CSS for that language, it should be used. Note that is only relevant for extension developers.
  • utils/TokenUtils
    • Method getModeAt has a hardcoded special case for XML. Once the other places are no longer based on this mode, but on the language, this can be removed to just report "xml". For HTML documents, the language manager maps the "xml" mode to the HTML language. XML documents are not affected by this. See issue #2965 for a related discussion.

Code that relies on the current editor state

These places currently access CodeMirror's state directly and are therefore not usable without an active editor. They might benefit from doing their own parsing, possibly using CodeMirror modes as parsers. CodeMirror's editor state could still optionally be used for optimization, but nothing else.

  • editor/EditorCommandHandlers.js
    • Functions _findCommentStart, _findCommentEnd and _findNextBlockComment use tokens provided by CodeMirror to search for comment boundaries. While the strings they search are provided by a language definition, this prevents us from defining arbitrary comment symbols. One example is "//" as the prefix for line comments (as SciTE does). Adding a comment this way is possible, but removing it does not work because "//" is not a prefix of the CodeMirror token for "//".
    • Methods blockCommentPrefixSuffix and lineCommentPrefixSuffix have similar constraints as they navigate by tokens instead of characters. In addition they check whether a token's className is different from "comment". Therefore they use tokens provided by CodeMirror.
  • language/CSSUtils.js
    • Method findMatchingRules to find CSS rules matching a selector. Searches an HTML document via language/HTMLUtils if it is the current full editor's document.
    • Method findSelectorAtDocumentPos to find the selector(s) of a CSS block, directly uses tokens provided by CodeMirror
    • Method getInfoAtPos to provide a context info object for the given cursor position, directly uses tokens provided by CodeMirror
  • language/HTMLUtils.js
    • Method findStyleBlocks to gather info about all <style> blocks in an HTML document, directly uses tokens provided by CodeMirror

Providing code semantics

This relates to places that require information about the semantics of code beyond what is provided by CodeMirror.

  • editor/Editor.js
    • Method _checkElectricChars adjusts the current line's indentation when blocks are ended. The characters to detect block boundaries are hard-coded - ], {, } and ). The function is supposed to replace CodeMirror's own implementation, citing bugs. In contrast to _checkElectricChars, CodeMirror's own implementation does not re-indent the line after typing "]" in a JavaScript file, so for now we cannot remove our own implementation without removing existing functionality.

Starting Live Preview

These areas are concerned with what files live preview can be started with. This also affects Brackets' behavior when switching between files while live preview is active.

  • file/FileUtils.js
    • Methods isStaticHtmlFileExt and isServerHtmlFileExt use hardcoded lists of file extensions to determine which file extensions are okay to open statically and which require a base URL to work. Switching to a file matching these criteria causes live preview to show that file instead unless it is included in the currently displayed file.
  • LiveDevelopment/LiveDevelopment.js
    • Method open reduces LiveDevelopment support to HTML files based on file extensions
    • Function _onDocumentChange closes LiveDevelopment when switching to a different, not included HTML file

Updating Live Preview

These areas are concerned with making sure that the live preview is up to date. This is needed if the main document or included files are changed.

  • LiveDevelopment/LiveDevelopment.js
    • Requires hardcoded list of special documents, namely {CSS,HTML,JS}Document. These function as updaters and highlighters.
    • Function _classForDocument uses a hardcoded mapping of file extensions to document types
    • Function _openDocument only loads related CSS documents (no JavaScript, not extensible)
    • Function _onLoad excludes CSSDocument from being marked as out of sync (not extensible)
    • Function _onDocumentChange excludes CSSDocument from being marked as out of sync (not extensible)
    • Function _onDocumentSaved only reloads the page if the document is not a CSSDocument (not extensible)
    • Function _onDirtyFlagChange only updates the LiveDevelopment status if the dirty file is not a CSSDocument (not extensible)

Showing the context in Live Preview

These areas are concerned with showing the context of the current cursor position by highlighting affected areas in the live preview and opening files related to elements in the page (GotoAgent).

  • LiveDevelopment/LiveDevelopment.js
    • Requires hardcoded list of special documents, namely {CSS,HTML,JS}Document. These function as updaters and highlighters.
    • Function _classForDocument uses a hardcoded mapping of file extensions to document types
    • Function _openDocument only loads related CSS documents (no JavaScript, not extensible)
    • Method showHighlight only calls doc.updateHighlight() for CSSDocuments
  • LiveDevelopment/Agents/GotoAgent.js
    • Has hardcoded support for HTML, CSS and JavaScript
  • LiveDevelopment/Agents/HighlightAgent
    • Has hardcoded support for HTML and CSS
  • LiveDevelopment/Agents/RemoteFunctions.js
    • Function _typeColor has hardcoded distinctions between html, css, js and others
Clone this wiki locally