Package babel :: Package messages :: Module jslexer |

# Module jslexer

A simple JavaScript 1.5 lexer which is used for the JavaScript extractor.

Classes
| Token Represents a token as returned by tokenize.

Functions
indicates_division(token) A helper function that helps the tokenizer to decide if the current token may be followed by a division operator. |
unquote_string(string) Unquote a string with JavaScript rules. |
tokenize(source) Tokenize a JavaScript source. |

Variables
| operators = ``[``'``>>>=``'``,` `'``>>>``'``,` `'``<<=``'``,` `'``>>=``'``,` `'``!=``'``,` `'``==``'``,` `'``<=``'``,` `'``>``...``
| escapes = ``{``'``b``'``:` `'``\x08``'``,` `'``f``'``:` `'``\x0c``'``,` `'``n``'``:` `'``\n``'``,` `'``r``'``:` `'``\r``'``,` `'``t``'``...``
| rules = ``[``(`None`,` re.compile(r'`(?u)`\s`+`')`)``,` `(`None`,` re.compile(r'<`...``
| division_re = `re.compile(r'/=`?`')`
| regex_re = `re.compile(r'`(?s)`/`(?:``[``^`/\\`]``*``(?:`\\.`[``^`/\\`]``*``)``*``)`/`[`a`-`zA`-``...``
| line_re = `re.compile(r'`(`\r\n`|`\n`|`\r`)`')`
| line_join_re = `re.compile(r'\\`(`\r\n`|`\n`|`\r`)`')`
| uni_escape_re = `re.compile(r'`[`a`-`fA`-`F0`-`9`]``{1,4}`')`
| __package__ = ``'``babel.messages``'``

Function Details

### unquote_string(string) |
Unquote a string with JavaScript rules. The string has to start with string delimiters (`'` or `"`.) Returns: a string

### tokenize(source) |
Tokenize a JavaScript source. Returns: generator of Tokens

Variables Details

  ### operators Value:
      ['>>>=',
       '>>>',
       '<<=',
       '>>=',
       '!=',
       '==',
       '<=',
       '>=',
      ...

  ### escapes Value:
      {'b': '\x08', 'f': '\x0c', 'n': '\n', 'r': '\r', 't': '\t'}

  ### rules Value:
      [(None, re.compile(r'(?u)\s+')),
       (None, re.compile(r'<!--.*')),
       ('linecomment', re.compile(r'//.*')),
       ('multilinecomment', re.compile(r'(?su)/\*.*?\*/')),
       ('name', re.compile(r'(?u)(\$+\w*|[^\W\d]\w*)')),
       ('number',
        re.compile(r'(?x)((?:0|[1-9]\d*)(\.\d+)?([eE][-\+]?\d+)?|(0x[a-fA-F0
      -9]+))')),
      ...

  ### regex_re Value:
      re.compile(r'(?s)/(?:[^/\\]*(?:\\.[^/\\]*)*)/[a-zA-Z]*')