Skip to content
Snippets Groups Projects
Commit 4e3202d2 authored by Jean-Philippe Lang's avatar Jean-Philippe Lang
Browse files

Reverts r3014 (CodeRay back to 0.7.6).

git-svn-id: svn+ssh://rubyforge.org/var/svn/redmine/trunk@3079 e93f8b46-1217-0410-a6f0-8f06a7374b81
parent d73fb1fa
No related branches found
No related tags found
No related merge requests found
Showing
with 758 additions and 216 deletions
= CodeRay - Trunk folder structure
== bench - Benchmarking system
All benchmarking stuff goes here.
Test inputs are stored in files named <code>example.<lang></code>.
Test outputs go to <code>bench/test.<encoder-default-file-extension></code>.
Run <code>bench/bench.rb</code> to get a usage description.
Run <code>rake bench</code> to perform an example benchmark.
== bin - Scripts
Executional files for CodeRay.
== demo - Demos and functional tests
Demonstrational scripts to show of CodeRay's features.
Run them as functional tests with <code>rake test:demos</code>.
== etc - Lots of stuff
Some addidtional files for CodeRay, mainly graphics and Vim scripts.
== gem_server - Gem output folder
For <code>rake gem</code>.
== lib - CodeRay library code
This is the base directory for the CodeRay library.
== rake_helpers - Rake helper libraries
Some files to enhance Rake, including the Autumnal Rdoc template and some scripts.
== test - Tests
Tests for the scanners.
Each language has its own subfolder and sub-suite.
Run with <code>rake test</code>.
This diff is collapsed.
= CodeRay
[- Tired of blue'n'gray? Try the original version of this documentation on
coderay.rubychan.de[http://coderay.rubychan.de/doc/] (use Ctrl+Click to open it in its own frame.) -]
http://rd.cYcnus.de/coderay/doc (use Ctrl+Click to open it in its own frame.) -]
== About
CodeRay is a Ruby library for syntax highlighting.
......@@ -18,11 +18,14 @@ And with line numbers.
* is what everybody should have on their website
* solves all your problems and makes the girls run after you
Version: 0.9.0
Version: 0.7.4 (2006.october.20)
Author:: murphy (Kornelius Kalnbach)
Contact:: murphy rubychan de
Website:: coderay.rubychan.de[http://coderay.rubychan.de]
License:: GNU LGPL; see LICENSE file in the main directory.
Subversion:: $Id: README 219 2006-10-20 15:52:25Z murphy $
-----
== Installation
......@@ -30,10 +33,17 @@ You need RubyGems[http://rubyforge.org/frs/?group_id=126].
% gem install coderay
Since CodeRay is still in beta stage, nightly buildy may be useful:
% gem install coderay -rs rd.cYcnus.de/coderay
=== Dependencies
CodeRay needs Ruby 1.8.6 or later. It also runs with Ruby 1.9.1+ and JRuby 1.1+.
CodeRay needs Ruby 1.8 and the
strscan[http://www.ruby-doc.org/stdlib/libdoc/strscan/rdoc/index.htm]
library (part of the standard library.) It should also run with Ruby 1.9 and
yarv.
== Example Usage
......@@ -50,9 +60,11 @@ CodeRay needs Ruby 1.8.6 or later. It also runs with Ruby 1.9.1+ and JRuby 1.1+.
See CodeRay.
Please report errors in this documentation to <murphy rubychan de>.
Please report errors in this documentation to <coderay cycnus de>.
-----
== Credits
=== Special Thanks to
......@@ -60,39 +72,30 @@ Please report errors in this documentation to <murphy rubychan de>.
* licenser (Heinz N. Gies) for ending my QBasic career, inventing the Coder
project and the input/output plugin system.
CodeRay would not exist without him.
* bovi (Daniel Bovensiepen) for helping me out on various occasions.
=== Thanks to
* Caleb Clausen for writing RubyLexer (see
http://rubyforge.org/projects/rubylexer) and lots of very interesting mail
traffic
* birkenfeld (Georg Brandl) and mitsuhiku (Arnim Ronacher) for PyKleur, now pygments.
You guys rock!
* birkenfeld (Georg Brandl) and mitsuhiku (Arnim Ronacher) for PyKleur. You
guys rock!
* Jamis Buck for writing Syntax (see http://rubyforge.org/projects/syntax)
I got some useful ideas from it.
* Doug Kearns and everyone else who worked on ruby.vim - it not only helped me
coding CodeRay, but also gave me a wonderful target to reach for the Ruby
scanner.
* everyone who uses CodeBB on http://www.rubyforen.de and http://www.python-forum.de
* iGEL, magichisoka, manveru, WoNáDo and everyone I forgot from rubyforen.de
* Dethix from ruby-mine.de
* zickzackw
* Dookie (who is no longer with us...) and Leonidas from http://www.python-forum.de
* everyone who used CodeBB on http://www.rubyforen.de and
http://www.infhu.de/mx
* iGEL, magichisoka, manveru, WoNDo and everyone I forgot from rubyforen.de
* Daniel and Dethix from ruby-mine.de
* Dookie (who is no longer with us...) and Leonidas from
http://www.python-forum.de
* Andreas Schwarz for finding out that CaseIgnoringWordList was not case
ignoring! Such things really make you write tests.
* closure for the first version of the Scheme scanner.
* Stefan Walk for the first version of the JavaScript scanner.
* Josh Goebel for another version of the JavaScript scanner and a Diff scanner.
* Jonathan Younger for pointing out the licence confusion caused by wrong LICENSE file.
* Jeremy Hinegardner for finding the shebang-on-empty-file bug in FileType.
* Charles Oliver Nutter and Yehuda Katz for helping me benchmark CodeRay on JRuby.
* Andreas Neuhaus for pointing out a markup bug in coderay/for_redcloth.
* 0xf30fc7 for the FileType patch concerning Delphi file extensions.
* The folks at redmine.org - thank you for using and fixing CodeRay!
* matz and all Ruby gods and gurus
* The inventors of: the computer, the internet, the true color display, HTML &
CSS, VIM, Ruby, pizza, microwaves, guitars, scouting, programming, anime,
CSS, VIM, RUBY, pizza, microwaves, guitars, scouting, programming, anime,
manga, coke and green ice tea.
Where would we be without all those people?
......@@ -100,27 +103,23 @@ Where would we be without all those people?
=== Created using
* Ruby[http://ruby-lang.org/]
* Chihiro (my Sony VAIO laptop); Henrietta (my old MacBook);
Triella, born Rico (my new MacBook); as well as
Seras and Hikari (my PCs)
* RDE[http://homepage2.nifty.com/sakazuki/rde_e.html],
VIM[http://vim.org] and TextMate[http://macromates.com]
* Subversion[http://subversion.tigris.org/]
* Redmine[http://redmine.org/]
* Firefox[http://www.mozilla.org/products/firefox/],
Firebug[http://getfirebug.com/], Safari[http://www.apple.com/safari/], and
* Chihiro (my Sony VAIO laptop), Henrietta (my new MacBook) and
Seras (my Athlon 2200+ tower)
* VIM[http://vim.org] and TextMate[http://macromates.com]
* RDE[http://homepage2.nifty.com/sakazuki/rde_e.html]
* Microsoft Windows (yes, I confess!) and MacOS X
* Firefox[http://www.mozilla.org/products/firefox/] and
Thunderbird[http://www.mozilla.org/products/thunderbird/]
* RubyGems[http://docs.rubygems.org/] and Rake[http://rake.rubyforge.org/]
* TortoiseSVN[http://tortoisesvn.tigris.org/] using Apache via
* Rake[http://rake.rubyforge.org/]
* RubyGems[http://docs.rubygems.org/]
* {Subversion/TortoiseSVN}[http://tortoisesvn.tigris.org/] using Apache via
XAMPP[http://www.apachefriends.org/en/xampp.html]
* RDoc (though I'm quite unsatisfied with it)
* Microsoft Windows (yes, I confess!) and MacOS X
* GNUWin32, MinGW and some other tools to make the shell under windows a bit
less useless
more useful
* Term::ANSIColor[http://term-ansicolor.rubyforge.org/]
* PLEAC[http://pleac.sourceforge.net/] code examples
=== Free
---
* As you can see, CodeRay was created under heavy use of *free* software.
* So CodeRay is also *free*.
......
#!/usr/bin/env ruby
# CodeRay Executable
#
# Version: 0.1
# Author: murphy
def err msg
$stderr.puts msg
end
begin
require 'coderay'
if ARGV.empty?
puts <<-USAGE
CodeRay #{CodeRay::VERSION} (http://rd.cYcnus.de/coderay)
Usage:
coderay -<lang> [-<format>] < file > output
coderay file [-<format>]
Example:
coderay -ruby -statistic < foo.rb
coderay codegen.c # generates codegen.c.html
USAGE
end
first, second = ARGV
if first
if first[/-(\w+)/] == first
lang = $1.to_sym
input = $stdin.read
tokens = :scan
elsif first == '-'
lang = $1.to_sym
input = $stdin.read
tokens = :scan
else
file = first
tokens = CodeRay.scan_file file
output_filename, output_ext = file, /#{Regexp.escape(File.extname(file))}$/
end
else
puts 'No lang/file given.'
exit 1
end
if second
if second[/-(\w+)/] == second
format = $1.to_sym
else
raise 'Invalid format (must be -xxx).'
end
else
$stderr.puts 'No format given; setting to default (HTML Page)'
format = :page
end
# TODO: allow streaming
if tokens == :scan
output = CodeRay::Duo[lang => format].highlight input #, :stream => true
else
output = tokens.encode format
end
out = $stdout
if output_filename
output_filename += '.' + CodeRay::Encoders[format]::FILE_EXTENSION
if File.exist? output_filename
err 'File %s already exists.' % output_filename
exit
else
out = File.open output_filename, 'w'
end
end
out.print output
rescue => boom
err "Error: #{boom.message}\n"
err boom.backtrace
err '-' * 50
err ARGV
exit 1
end
#!/usr/bin/env ruby
require 'coderay'
puts CodeRay::Encoders[:html]::CSS.new.stylesheet
# = CodeRay Library
#
# $Id: coderay.rb 227 2007-04-24 12:26:18Z murphy $
#
# CodeRay is a Ruby library for syntax highlighting.
#
# I try to make CodeRay easy to use and intuitive, but at the same time fully featured, complete,
......@@ -105,7 +107,7 @@
#
# CodeRay.scan_stream:: Scan in stream mode.
#
# == All-in-One Encoding
# == All-in-One Encoding
#
# CodeRay.encode:: Highlight a string with a given input and output format.
#
......@@ -128,14 +130,13 @@
module CodeRay
# Version: Major.Minor.Teeny[.Revision]
# Major: 0 for pre-stable, 1 for stable
# Minor: feature milestone
# Teeny: development state, 0 for pre-release
# Revision: Subversion Revision number (generated on rake gem:make)
VERSION = '0.9.0'
# Major: 0 for pre-release
# Minor: odd for beta, even for stable
# Teeny: development state
# Revision: Subversion Revision number (generated on rake)
VERSION = '0.7.6'
require 'coderay/tokens'
require 'coderay/token_classes'
require 'coderay/scanner'
require 'coderay/encoder'
require 'coderay/duo'
......@@ -314,7 +315,6 @@ end
# Run a test script.
if $0 == __FILE__
$stderr.print 'Press key to print demo.'; gets
# Just use this file as an example of Ruby code.
code = File.read(__FILE__)[/module CodeRay.*/m]
print CodeRay.scan(code, :ruby).html
end
......@@ -2,6 +2,8 @@ module CodeRay
# = Duo
#
# $Id: scanner.rb 123 2006-03-21 14:46:34Z murphy $
#
# A Duo is a convenient way to use CodeRay. You just create a Duo,
# giving it a lang (language of the input code) and a format (desired
# output format), and call Duo#highlight with the code.
......
require "stringio"
module CodeRay
# This module holds the Encoder class and its subclasses.
......@@ -130,56 +132,30 @@ module CodeRay
# By default, it calls text_token or block_token, depending on
# whether +text+ is a String.
def token text, kind
encoded_token =
if text.is_a? ::String
out =
if text.is_a? ::String # Ruby 1.9: :open.is_a? String
text_token text, kind
elsif text.is_a? ::Symbol
block_token text, kind
else
raise 'Unknown token text type: %p' % text
end
append_encoded_token_to_output encoded_token
end
def append_encoded_token_to_output encoded_token
@out << encoded_token if encoded_token && defined?(@out) && @out
@out << out if @out
end
# Called for each text token ([text, kind]), where text is a String.
def text_token text, kind
end
# Called for each block (non-text) token ([action, kind]), where action is a Symbol.
def block_token action, kind
case action
when :open
open_token kind
when :close
close_token kind
when :begin_line
begin_line kind
when :end_line
end_line kind
else
raise 'unknown block action: %p' % action
end
end
# Called for each block token at the start of the block ([:open, kind]).
def open_token kind
end
# Called for each block token end of the block ([:close, kind]).
def close_token kind
end
# Called for each line token block at the start of the line ([:begin_line, kind]).
def begin_line kind
end
# Called for each line token block at the end of the line ([:end_line, kind]).
def end_line kind
end
# Called with merged options after encoding starts.
# The return value is the result of encoding, typically @out.
......@@ -191,16 +167,8 @@ module CodeRay
#
# The already created +tokens+ object must be used; it can be a
# TokenStream or a Tokens object.
if RUBY_VERSION >= '1.9'
def compile tokens, options
for text, kind in tokens
token text, kind
end
end
else
def compile tokens, options
tokens.each(&self)
end
def compile tokens, options
tokens.each(&self)
end
end
......
......@@ -35,14 +35,6 @@ module Encoders
">"
end
def begin_line kind
"#{kind}["
end
def end_line kind
"]"
end
end
end
......
......@@ -9,9 +9,10 @@ module Encoders
register_for :div
DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge \
DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge({
:css => :style,
:wrap => :div
:wrap => :div,
})
end
......
......@@ -25,6 +25,10 @@ module Encoders
#
# == Options
#
# === :escape
# Escape html entities
# Default: true
#
# === :tab_width
# Convert \t characters to +n+ spaces (a number.)
# Default: 8
......@@ -41,12 +45,6 @@ module Encoders
#
# Default: nil
#
# === :title
#
# The title of the HTML page (works only when :wrap is set to :page.)
#
# Default: 'CodeRay output'
#
# === :line_numbers
# Include line numbers in :table, :inline, :list or nil (no line numbers)
#
......@@ -62,16 +60,6 @@ module Encoders
#
# Default: 10
#
# === :highlight_lines
#
# Highlights certain line numbers now by using the :highlight_lines option.
# Can be any Enumerable, typically just an Array or Range, of numbers.
#
# Bolding is deactivated when :highlight_lines is set. It only makes sense
# in combination with :line_numbers.
#
# Default: nil
#
# === :hint
# Include some information into the output using the title attribute.
# Can be :info (show token type on mouse-over), :info_long (with full path)
......@@ -86,19 +74,19 @@ module Encoders
FILE_EXTENSION = 'html'
DEFAULT_OPTIONS = {
:escape => true,
:tab_width => 8,
# :level => :xhtml, # reserved for future use
:level => :xhtml,
:css => :class,
:style => :cycnus,
:wrap => nil,
:title => 'CodeRay output',
:line_numbers => nil,
:line_number_start => 1,
:bold_every => 10,
:highlight_lines => nil,
:hint => false,
}
......@@ -153,7 +141,7 @@ module Encoders
when :debug
classes.inspect
end
title ? " title=\"#{title}\"" : ''
" title=\"#{title}\""
end
def setup options
......@@ -162,6 +150,7 @@ module Encoders
@HTML_ESCAPE = HTML_ESCAPE.dup
@HTML_ESCAPE["\t"] = ' ' * options[:tab_width]
@escape = options[:escape]
@opened = [nil]
@css = CSS.new options[:style]
......@@ -175,7 +164,7 @@ module Encoders
when :class
@css_style = Hash.new do |h, k|
c = CodeRay::Tokens::ClassOfKind[k.first]
c = Tokens::ClassOfKind[k.first]
if c == :NO_HIGHLIGHT and not hint
h[k.dup] = false
else
......@@ -233,70 +222,43 @@ module Encoders
@out.css = @css
@out.numerize! options[:line_numbers], options
@out.wrap! options[:wrap]
@out.apply_title! options[:title]
super
end
def token text, type = :plain
case text
when nil
# raise 'Token with nil as text was given: %p' % [[text, type]]
when String
if text =~ /#{HTML_ESCAPE_PATTERN}/o
def token text, type
if text.is_a? ::String
if @escape && (text =~ /#{HTML_ESCAPE_PATTERN}/o)
text = text.gsub(/#{HTML_ESCAPE_PATTERN}/o) { |m| @HTML_ESCAPE[m] }
end
@opened[0] = type
if text != "\n" && style = @css_style[@opened]
if style = @css_style[@opened]
@out << style << text << '</span>'
else
@out << text
end
# token groups, eg. strings
when :open
@opened[0] = type
@out << (@css_style[@opened] || '<span>')
@opened << type
when :close
if @opened.empty?
# nothing to close
else
if $DEBUG and (@opened.size == 1 or @opened.last != type)
raise 'Malformed token stream: Trying to close a token (%p) \
that is not open. Open are: %p.' % [type, @opened[1..-1]]
else
case text
when :open
@opened[0] = type
@out << (@css_style[@opened] || '<span>')
@opened << type
when :close
if @opened.empty?
# nothing to close
else
if $DEBUG and (@opened.size == 1 or @opened.last != type)
raise 'Malformed token stream: Trying to close a token (%p) \
that is not open. Open are: %p.' % [type, @opened[1..-1]]
end
@out << '</span>'
@opened.pop
end
@out << '</span>'
@opened.pop
end
# whole lines to be highlighted, eg. a deleted line in a diff
when :begin_line
@opened[0] = type
if style = @css_style[@opened]
@out << style.sub('<span', '<div')
when nil
raise 'Token with nil as text was given: %p' % [[text, type]]
else
@out << '<div>'
raise 'unknown token kind: %p' % text
end
@opened << type
when :end_line
if @opened.empty?
# nothing to close
else
if $DEBUG and (@opened.size == 1 or @opened.last != type)
raise 'Malformed token stream: Trying to close a line (%p) \
that is not open. Open are: %p.' % [type, @opened[1..-1]]
end
@out << '</div>'
@opened.pop
end
else
raise 'unknown token kind: %p' % [text]
end
end
......
......@@ -27,19 +27,16 @@ module Encoders
1.upto(styles.size) do |offset|
break if style = cl[styles[offset .. -1]]
end
$stderr.puts 'Style not found: %p' % [styles] if $DEBUG and style.empty?
raise 'Style not found: %p' % [styles] if $DEBUG and style.empty?
return style
end
private
CSS_CLASS_PATTERN = /
( # $1 = selectors
(?:
(?: \s* \. [-\w]+ )+
\s* ,?
)+
)
( (?: # $1 = classes
\s* \. [-\w]+
)+ )
\s* \{ \s*
( [^\}]+ )? # $2 = style
\s* \} \s*
......@@ -47,14 +44,12 @@ module Encoders
( . ) # $3 = error
/mx
def parse stylesheet
stylesheet.scan CSS_CLASS_PATTERN do |selectors, style, error|
stylesheet.scan CSS_CLASS_PATTERN do |classes, style, error|
raise "CSS parse error: '#{error.inspect}' not recognized" if error
for selector in selectors.split(',')
classes = selector.scan(/[-\w]+/)
cl = classes.pop
@classes[cl] ||= Hash.new
@classes[cl][classes] = style.to_s.strip.delete(' ').chomp(';')
end
styles = classes.scan(/[-\w]+/)
cl = styles.pop
@classes[cl] ||= Hash.new
@classes[cl][styles] = style.to_s.strip
end
end
......
......@@ -32,19 +32,9 @@ module Encoders
#end
bold_every = options[:bold_every]
highlight_lines = options[:highlight_lines]
bolding =
if bold_every == false && highlight_lines == nil
if bold_every == false
proc { |line| line.to_s }
elsif highlight_lines.is_a? Enumerable
highlight_lines = highlight_lines.to_set
proc do |line|
if highlight_lines.include? line
"<strong class=\"highlighted\">#{line}</strong>" # highlighted line numbers in bold
else
line.to_s
end
end
elsif bold_every.is_a? Integer
raise ArgumentError, ":bolding can't be 0." if bold_every == 0
proc do |line|
......@@ -61,12 +51,12 @@ module Encoders
case mode
when :inline
max_width = (start + line_count).to_s.size
line_number = start
line = start
gsub!(/^/) do
line_number_text = bolding.call line_number
indent = ' ' * (max_width - line_number.to_s.size) # TODO: Optimize (10^x)
res = "<span class=\"no\">#{indent}#{line_number_text}</span> "
line_number += 1
line_number = bolding.call line
indent = ' ' * (max_width - line.to_s.size)
res = "<span class=\"no\">#{indent}#{line_number}</span> "
line += 1
res
end
......@@ -75,12 +65,12 @@ module Encoders
# Because even monospace fonts seem to have different heights when bold,
# I make the newline bold, both in the code and the line numbers.
# FIXME Still not working perfect for Mr. Internet Exploder
# FIXME Firefox struggles with very long codes (> 200 lines)
line_numbers = (start ... start + line_count).to_a.map(&bolding).join("\n")
line_numbers << "\n" # also for Mr. MS Internet Exploder :-/
line_numbers.gsub!(/\n/) { "<tt>\n</tt>" }
line_numbers_table_tpl = TABLE.apply('LINE_NUMBERS', line_numbers)
gsub!(/<\/div>\n/) { '</div>' }
gsub!(/\n/) { "<tt>\n</tt>" }
wrap_in! line_numbers_table_tpl
@wrapped_in = :div
......@@ -100,9 +90,8 @@ module Encoders
end
close = '</span>' * opened_tags.size
"<li>#{open}#{line}#{close}</li>\n"
"<li>#{open}#{line}#{close}</li>"
end
chomp!("\n")
wrap_in! LIST
@wrapped_in = :div
......
......@@ -86,11 +86,6 @@ module Encoders
Template.wrap! self, template, 'CONTENT'
self
end
def apply_title! title
self.sub!(/(<title>)(<\/title>)/) { $1 + title + $2 }
self
end
def wrap! element, *args
return self if not element or element == wrapped_in
......@@ -105,10 +100,6 @@ module Encoders
wrap! :div if wrapped_in? nil
raise "Can't wrap %p in %p" % [wrapped_in, element] unless wrapped_in? :div
wrap_in! Output.page_template_for_css(@css)
if args.first.is_a?(Hash) && title = args.first[:title]
apply_title! title
end
self
when nil
return self
else
......@@ -175,9 +166,7 @@ module Encoders
# title="double click to expand"
LIST = <<-`LIST`
<ol class="CodeRay">
<%CONTENT%>
</ol>
<ol class="CodeRay"><%CONTENT%></ol>
LIST
PAGE = <<-`PAGE`
......@@ -186,7 +175,7 @@ module Encoders
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="de">
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title></title>
<title>CodeRay HTML Encoder Example</title>
<style type="text/css">
<%CSS%>
</style>
......
......@@ -9,10 +9,11 @@ module Encoders
register_for :page
DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge \
DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge({
:css => :class,
:wrap => :page,
:line_numbers => :table
})
end
......
......@@ -9,9 +9,10 @@ module Encoders
register_for :span
DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge \
DEFAULT_OPTIONS = HTML::DEFAULT_OPTIONS.merge({
:css => :style,
:wrap => :span
:wrap => :span,
})
end
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment