Pyjama Development Summer 2010

From IPRE Wiki
Jump to: navigation, search

May 26, 2010

Have easily installed Pyjama for my Windows 7 Home Edition 32bit edition. Currently still working on installing it for a Fedora distribution. The mono installation worked, but currently having issues installing Pyjama due to the libgdiplus reference not working. Will continue to investigate the problem.

  • Update!
    • Maybe it is not the libgdiplus, but the .resx files compiled using resgen2?

Have yet to install Pyjama on Mac 10.X distribution since waiting for Computing Services to set up a switch board for all three operating systems to run on single monitor. The next few days will be studying and becoming familiar with the code and research/investigate bugs/fixes in Pyjama.

Concerns

Currently working off Pyjama on Windows 7 OS. Just from looking at images on previous web pages regarding Pyjama and how it works, I have found some misleading images that might confuse students/educators who would like to use Pyjama. Will create new screen shots to reflect accurate details.

Start Up

  • Opens empty tab/file -- currently unsaved
  • Starts off on Python mode (default)
  • Help is currently empty
  • In document, lack of automatic tabbing
  • In Options - Configure, the user can change size of font.
  • In Shell - Evaluate, user can select Enter and Newline? Not exactly sure how this fits.
  • In Edit - Select Editor, has no functionality.
  • In File - Print (have not tried to see if it works yet)

Suggestions

  • When switching from a *.py to *.rb or any environment, need to also manually switch control to language on bottom of Pyjama window. Would like that to be automatic.
  • In Options - Configure, allow changes in background and font color.


May 27, 2010

Have Pyjama installed on Linux computer. Currently reading code snippets and doing some studying regarding winforms (different from WPF). Currently reading up on FIXME code snippets and how to make the code cleaner and more readable

Start Up

  • In Pyjama editor, when selecting the Language drop down menu, no functionality.
  • In Menu - Shell tab, not exactly sure what select shell does other than move cursor to Pyjama shell window.
  • In Pyjama shell, the underscore "_" is not visible until clicked.

Work List

  • Update installation instructions for Pyjama for Linux distributions.
  • FIXME 1: Generalize code to select Language in Pyjama shell.
  • FIXME 2: When opening code file, change editor and shell to reflect programming environment.
  • Create new screen shots to reflect accurate functionality of Pyjama.
  • FIXME 3: Expand configure options for user <need to do research on graphics>
  • Install Pyjama for Mac 10.X (will look up version later).
  • FIXME 4: Change the way the file is parsed to reflect programming environment. For example, keywords should be blue, quotes(single, double, triple) should be unique, and comments should be red. Note: The keyword self (should it be highlighted? In IDLE, it is not.)

June 1, 2010

Today I will be researching how to write *.config files (looks like its in XML) and how to read from them. The reason for this *.config file is to populate all the keywords for Python and Ruby so that these keywords are highlighted. Note that "self" should maybe be highlighted, but not sure yet. Also, need to find an efficient way to parse a *.py or *.rb file (since built in method Streamreader is very slow). Maybe need to write the parse code in C, but will find ways to keep same programming language (C#).

June 2, 2010

FIXME 1 is complete. Made the code more elegant and readable. Working on attempting to read from *.config file to make chances to color scheme.

MAJOR GOAL:

  • To have Scribbler robots up and running using Pyjama.

Suggestions

  • Wouldn't it be nice to be able to clear the Pyjama Shell. Personally, I have always been annoyed with the Python shell in not allowing a user to clear the screen (there is no clear() command in Python), so it would be nice to have that option in Pyjama.
  • Need to differentiate between F5 (Run) and the button Run! on Pyjama shell since pressing F5 restarts the shell and Run! executes a command.
  • Maybe remove Run! button and replace with Clear button. :)

June 9, 2010

In the next few days I will need to think how to parse a file by loading a *.config file for Python/Ruby. Need to think how to parse special strings like ";" or "<" in XML (might not be able to do it in XML). It might be that I will need to rewrite the parser entirely to work with the XML config code).

  • Ideas on how to access *.config files

Now, the idea is to parse a file (Python or Ruby) and load the configuration file associated with the programming environment. For example, keywords like "pass" and "class" will be of one color, and "True" and "False" will be of another color.

Also, we would like for users to edit the configuration such that if they do not like the keywords to be of one color, they can change it. Thus, as a programmer, we need to be able to save changes to the configuration (by not over-writing the original), and the user can have the option to save it to file and load it as well.

For now, concentrate on loading *.config file for programming environment (default: Python)

  • Algorithmic Process Flow (tentative)
    • Access *.config file from event handler
    • Read keywords in from *.config file to dictionary (ex: key="pass" value="orange")
    • Read all settings as well (like font size, quote styles, and defaults)
    • Parse *.py file (or *.rb)
      • Encounter keywords in dictionary, associate color with it

June 10, 2010

  • New Concepts/Ideas
    • There exists some editors that allow automatic correction/type settings when the user types code. Recent version of Python 2.5 and above and Visual Studio have this feature. It would be a wonderful feature to include in Pyjama, especially for students who cannot remember specific keywords. Also, this feature would include automatic correction, which can prevent users from misspelling or incorrectly implementing code.
    • Now, there exists many ways to access a *.config file. The method I will be using is under the System.XML namespace, which is not as strict as the System.Configuration namespace.
  • Issues/Bugs
    • Interesting enough, when Pyjama.exe is compiled, the Python.config file is not copied to the /bin/Debug folder, where other references/Pyjama.exe is located. This causes a disturbance in the force, or more like an hassle to find the relative path of the file.

June 11, 2010

Added Python.config file to Pyjama. Now all configurations like color coding is set by an XML file. Now need to create a Ruby config file which will be similar to the Python, and determine how that will run next week.

June 14, 2010

Now that we have a working configuration file on hand (just Python for now), we need to work on speeding up the parsing process. Currently, the parsing is done character by character, but I learned from my algorithms class that is we can use regular expressions to parse a LINE, not character by character, we might be able to create an efficient syntax highlighting tool. For C#, there exists a RegEx, a regular expression class that might be able to help in this situation. Also, we can parse a Python file line by line (will prove difficult for commented code that can span to multiple lines; we can make this a special case). I will pursue this idea some more and see if this can prove to be the solution we need.

Also, since another addition to the syntax highlighting will be auto-completion.

June 15, 2010

Today I fixed a portion of a bug in the Pyjama editor. In the past, when a user opens a file, the environment's default is the Python programming environment. However, when the user switches over to another language, like Ruby, the programming environment remains in Python mode.

The partial fix changes the console window when the user runs the code (by pressing F5). I'm looking into and deciding if when a user opens a file, should the editor also reflect that environment (it should since Python and Ruby have different syntax coloring).

June 16, 2010

New Fix: Was able to load *.config file based on open file. Default is the Python configuration file. During the loading process, in the DocumentPage protocol, a file extension was being passed in, but files being read in from the Open File Menu did not grab the extension, thus returning a null value. I changed the

DocumentPage(IMainForm main_form, ActiveCodeFile file) : this(main_form, file.FileExtension)

to

DocumentPage(IMainForm main_form, ActiveCodeFile file) : this(main_form, file.FileName)

Then, I created a new function called GetConfigFile(string filename), which retrieves the file extension of filename and returns the specific configuration file to be loaded into the Pyjama shell.

Question: Should the language environment change automatically when a file is opened or when it is running?

Bug Caught When editing a *.cs file in Pyjama, Pyjama unexpectedly quit with this error message.

System.FormatException: Could not find any parsable digits.
at System.Convert.ConvertFromBase (System.String value, Int32 fromBase, Boolean unsigned) [0x001e9] in /builddir/build/BUILD/mono-2.4.3.1/mcs/class/corlib/System/Convert.cs:2607 
at System.Convert.ToByte (System.String value, Int32 fromBase) [0x00000] in /builddir/build/BUILD/mono-2.4.3.1/mcs/class/corlib/System/Convert.cs:525 
at System.Windows.Forms.RTF.RTF.GetToken2 () [0x00126] in /builddir/build/BUILD/mono-2.4.3.1/mcs/class/Managed.Windows.Forms/System.Windows.Forms.RTF/RTF.cs:483 
at System.Windows.Forms.RTF.RTF.GetToken () [0x0004a] in /builddir/build/BUILD/mono-2.4.3.1/mcs/class/Managed.Windows.Forms/System.Windows.Forms.RTF/RTF.cs:369 
at System.Windows.Forms.RTF.RTF.Read () [0x0000b] in /builddir/build/BUILD/mono-2.4.3.1/mcs/class/Managed.Windows.Forms/System.Windows.Forms.RTF/RTF.cs:311 
at System.Windows.Forms.RichTextBox.InsertRTFFromStream (System.IO.Stream data, Int32 cursor_x, Int32 cursor_y, System.Int32& to_x, System.Int32& to_y, System.Int32& chars) [0x00110] in /builddir/build/BUILD/mono-2.4.3.1/mcs/class/Managed.Windows.Forms/System.Windows.Forms/RichTextBox.cs:1838 
at System.Windows.Forms.RichTextBox.InsertRTFFromStream (System.IO.Stream data, Int32 cursor_x, Int32 cursor_y) [0x00000] in /builddir/build/BUILD/mono-2.4.3.1/mcs/class/Managed.Windows.Forms/System.Windows.Forms/RichTextBox.cs:1805 
at System.Windows.Forms.RichTextBox.set_Rtf (System.String value) [0x0001d] in /builddir/build/BUILD/mono-2.4.3.1/mcs/class/Managed.Windows.Forms/System.Windows.Forms/RichTextBox.cs:360 
at Pyjama.MyRichTextBox.WndProc (System.Windows.Forms.Message& m) [0x0004c] in /home/msbeard/Pyjama/src/Controls/Document.cs:35 
at System.Windows.Forms.Control+ControlWindowTarget.OnMessage (System.Windows.Forms.Message& m) [0x00000] in /builddir/build/BUILD/mono-2.4.3.1/mcs/class/Managed.Windows.Forms/System.Windows.Forms/Control.cs:234 
at System.Windows.Forms.Control+ControlNativeWindow.WndProc (System.Windows.Forms.Message& m) [0x00000] in /builddir/build/BUILD/mono-2.4.3.1/mcs/class/Managed.Windows.Forms/System.Windows.Forms/Control.cs:215 
at System.Windows.Forms.NativeWindow.WndProc (IntPtr hWnd, Msg msg, IntPtr wParam, IntPtr lParam) [0x00085] in /builddir/build/BUILD/mono-2.4.3.1/mcs/class/Managed.Windows.Forms/System.Windows.Forms/NativeWindow.cs:242

Fixed relative and absolute path of *.config files for programming environment. Determined that config files will either be in one or two directories. Either the bin/Debug/Config folder or Pyjama/src/Config folder. Interesting enough, when Pyjama is run initially, config file is loaded from /bin/Debug/Config folder. After a file is compiled and ran, the directory for searching the config files is in the Pyjama/src/Config folder.


Need to edit Document.cs to read in the different methods to comment out text (since they differ with each programming language).


June 17, 2010

Today I will be concentrating on fixing the syntax highlighting for double and triple quotes. This however is difficult since we are not only working with Python, but also Ruby and C# syntax highlighting. My idea is to truly rely on the *.config files of the specific programming language in order to do syntax highlighting correctly, especially for commenting and quotes. For example, Ruby does not have any comments related to triple quotes and the symbol # like in Python, so Pyjama should not comment code in triple quotes if in Ruby mode.

The second major issue is to make syntax highlighting faster and more efficient. The current implementation is very slow on some slower computer models. Thus, I will try again to implement an efficient parsing method using regular expressions (which is supported in mono).

Today's Accomplishments

  • Learned regular expression syntax.
  • Coded all necessary patterns needed for parsing.
  • Learned a Rich Text Box cannot render HTML (boo).

June 18, 2010

Efficient syntax highlighting has yet to be conquered. Through extensive research of different implementations of syntax highlighting, I have yet to find one (and the current implementation) accurate and quick.

The version I have written only colors the first line. My idea/algorithm, was to split each line into words, and then lookup each word into a dictionary. Note: a word surrounded by quotations will be one word, thus supposedly making syntax highlighting for quotations easier. However, comments will need to be treated differently.

The original version searches character by character, and my implementation goes word by word.

Either way, using Regex expressions or searching by characters is not solving the inefficiency or accuracy of our problem. Maybe the best solution would be to highlight only what is visible to the user, and forgo syntax highlighting the entire document.

Next Week

  • Robots in Pyjama
  • Graphics in Pyjama

Note on Graphics in Pyjama

  • Looks like only for Python
  • Check out Zelle's Graphics Library
  • Try to implement this in C#

June 21, 2010

Graphics in Pyjama

  • The idea is to write a graphics library similar to John Zelle's implementation in Python.
    • His library inherits from the TK class which is another Python graphics library to draw objects in a window
  • For this project for Pyjama, we would like to write a C# graphics library that can be run in both IronPython and IronRuby.
    • This graphics library will inherit from the .NET Winforms/WPF class to simply display graphic objects like points, circles, etc.
  • We will use Zelle's graphics library as a model to study and implement this graphical feature for Pyjama, which will allow students and educators to run graphical code in Python and Ruby to display graphics.

Steps to Design and Implement Graphics Library

  • Understand Zelle's graphics class
  • Threads (new)
  • OO programming (learned but never hurts to review)
  • Winforms/WPF (more used to WPF since that is where Microsoft is moving towards/ Winforms however will probably always be supported)

Features of C# Graphics Library

  • Geometry objects (point, line, circle, rectangle, polygon, etc)
  • Winform objects (button, text box, menu)
  • Images (what formats are supported?)
  • Text
  • Transformations (move, undraw, flip, etc)

How to call graphics library

  • Python running in Pyjama
 from graphics import *
 win = GraphiWin("Draw Circle", 200, 200)
 center = Point(100, 100)
 circ = Circle(center, 30)
 circ.draw(win)
  • Same code in both Python's IDLE and Python running in Pyjama should display the same image. Thus, students can run their code in Pyjama and move it to Pythons IDLE (which Zelle's graphics library is installed) and render a similar window with objects inside.

June 22/23, 2010

The goal of this week is to get ipgraphics.py, a Python graphics class, to be written in C# so that we can have graphics running for both Python and Ruby in Pyjama. I knew I would have to review OOP and constructors for C#, so that is what I have been studying and writing code for. I also have been learning about constructors and how to overload them in C#.

Example 1: Python Constructor Overloading

I declare a class called Window with three constructors: title, width, and height.

 class Window:
  def __init__(self, title="Graphics Window", width=200, height=200):
   self.title= title
   self.height = height
   self.width = width

In Python IDLE or Pyjama:

  • Declare Window instance
 win = Window()
 win.title  ==> "Graphics Window"
 win.width  ==> 200
 win.height ==> 200

Overloading

  • Now lets overload the constructors and change the title, width, and height of the window.
 win = Window("My Window", 500, 40)
 win.title  ==> "My Window"
 win.width  ==> 500
 win.height ==> 40
  • We can also just overload one or more of the constructors. In this example, lets overload the title constructor.
 win = Window("My Window")
 win.title  ==> "My Window"
 win.width  ==> 200
 win.height ==> 200
  • Note that the width and height constructor returned the default values defined in the class Window.

Example 2: C# Constructor Overloading

To do the same thing in C#, it requires much more code to specify the constructor.

 using System;
 namespace graphics
 {
    public class Window
    {
        public string Title;
        public int Width;
        public int Height;
        public Window()
        {
            Title = "Graphics Window";
            Width = 200;
            Height = 200;
        }
        public Window(string title)
        {
            Title = title;
            Width = 200;
            Height = 200;
        }
        public Window(string title, int height)
        {
            Title = title;
            Height = height;
            Width = 200;
        }
        public Window(string title, int height, int width)
        {
            Title = title;
            Height = height;
            Width = width; 
        }
        static void Main()
        {
            Window win = new Window("My Window", 500);
            Console.WriteLine("Window Properties");
            Console.WriteLine("Title =  {0}", win.Title);  ==> "My Window"
            Console.WriteLine("Height = {0}", win.Height); ==> 500
            Console.WriteLine("Width =  {0}", win.Width);  ==> 200
            Console.Read();
        }
    } 
 }

I believe there is a much easier way to declare constructors using get/set, and I will look into this.

June 28, 2010

Today's goal was to learn about threading. In this section, I'm going to present an overview of what threads are and how it can be applied in C#.

General

Threading allows a program to do concurrent processing so you can do more that one operation at a time. Unlike parallelism, threads share the same CPU and only run one thread at a time. However, the CPU switches back and forth between threads, mimicking what looks like multiple processes running, but in reality one running every few seconds, or however it is scheduled.

In graphics, we would like to create windows to be drawn, and it can be a single window or multiple windows to be running on different threads. Thus, if we have two windows open, each will be running on a different thread. Each thread shares resources, and if one dies, others will continue running. True parallel processing does not allow that (or makes it difficult to do so).

Why Threads?

  • Multi-tasking
  • Efficiency

Threading in C#

Threading in C# is a good resource for learning about threading in C# and has helped me review/learn more about threads and its similarities and difference to parallel processing.

July 1, 2010

So this past week I have been steadily writing the graphics module for Pyjama. This graphics class inherits mostly form System.Drawing.Graphics and the window itself inherits from System.Windows.Forms.

The major problem I have come across in developing this module is the _draw(Graphics g) method in the base class Graphics Object. The _draw() method is a generic drawing class which is written over in a child class, like Point for example.

In Python, it is not explicit how a derived class overrides methods of their base class. For example, lets say we have a parent class Animal, and that the Dog and Cat class are children of the base class Animal.

In Python

class Animal:
    def __init__(self):
        pass
    
    def talk(self):
        print "I'm an animal!"

class Dog(Animal):
    def __init__(self):
        pass
    
    def talk(self):
        print "Woof!"

class Cat(Animal):
    def __init_(self):
        pass

    def talk(self):
        print "Meow!"

So when we invoke the Cat and Dog talk() methods, we end up with:

>>> d = Dog()
>>> d.talk()
Woof!
>>> c = Cat()
>>> c.talk()
Meow!
>>> a = Animal()
>>> a.talk()
I'm an animal!

Now, to do the same thing in C# is similar, yet not as vague as Python makes it seem.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Animal
{
    public class Animal
    {
        public virtual void talk()
        {
            Console.WriteLine("I'm an animal!");
        }
    }

    public class Dog : Animal
    {
        public override void talk()
        {
            Console.WriteLine("Woof!");
        }
    }

    public class Cat : Animal
    {
        public override void talk()
        {
            Console.WriteLine("Meow!");
        }
    }

    class Program
    {
        static void Main(string[] args)
        {
            Animal a = new Animal();
            a.talk();

            Dog d = new Dog();
            d.talk();

            Cat c = new Cat();
            c.talk();
        }
    }
}
I'm an animal!
Woof!
Meow!

Notice how we explicitly override the talk() method in the derived Dog and Cat classes by using the keyword override. Also notice the keyword virtual. In C#, if we want to modify a method and allow it to be overridden in a derived class, we use the keyword virtual.

July 9, 2010

Quick update

Nearly done with the graphics module in C#. Found a bug in the Entry class yesterday and am working on fixing that now. I need to also translate the serial module next, but that will be more difficult since the serial module inherits from another serial packing written in Python. I'm at a lost on how to do this since all the code is in Python, so it might require more creative measures to translate the language. It might not be so bad when I analyze the IronPython serial module and the Python serial package.

July 16, 2010

How fast can the computer draw

Decided to determine how fast the graphics module is in creating a graphics object and drawing it on a window. I compared two graphics modules, the ipgraphics module (IRONPython) and the graphics module (C#). I determined that the C# graphics module is much faster than the ipgraphics module. For example, creating 5000 number of points and drawing them using the graphics module took .2185 seconds, while the same number of points drawn using the ipgraphics module took about 100 seconds.

Also, I think I found a major bug in the syntax highlighting in Pyjama. It has not been verified yet, but I think that the code:

  • does not remove white space accurately
  • the parser looks at each line of code more than once (at least 5 times)

July 30, 2010

Major Changes to Pyjama:

  • Faster Syntax Highlighting

In previous versions of Pyjama, opening a file and highlighting all of the syntax was a computationally expensive process. I originally thought it was the FormatAll() function (the code to syntax highlight) that was the cause of the slow down. However, analyzing the code and the way a TextBox behaves, I have come to realize that FormatAll() was being called more than once.

When reading a file into the TextBox object, a TextBox_Changed handler is fired, but it is not fired once, but fired every time a new character is added to the Textbox. I originally thought the number of times FormatAll() would be called is once per line, but that is untrue. For example, I measured the speed and the number of times FormatAll() was executed during the initial setup of the main Pyjama form. This is where the TextBox_Changed handler was created. In reading a Document.cs file which contains 546 lines of code, FormatAll() was called 9778 times, and the execution time to syntax highlight took 34.18 seconds, running under my personal laptop. Another document, graphics.cs, contains 1475 lines of code. FormatAll() was called 9561 times, and the execution time for syntax highlighting took 154.13 seconds (< 3 min.).

Now, I changed where the TextBox_Changed handler was to be created. After the initial reading of the file, I added the TextBox_Changed handler. Now, in reading the same document mentioned above, FormatAll() was called 546 times (same as the number of lines) and the execution time to syntax highlight took 2.39 seconds, running on my personal laptop. For the graphics.cs document, FormatAll() was executed 1475 times (same as the number of lines), and syntax highlighting execution took about 20 seconds on my personal laptop.

  • Configuration Files

In previous versions of Pyjama, the syntax highlighting elements (font color, size, and style) was hard coded to fit a Python syntax highlighting style. In C#, you are able to create configuration files (written in xml), and there exists an API to read XML files and to grab important data from them. Thus, I created a Python, Ruby, and C# syntax highlighting configuration file that is loaded into Pyjama when a specific file extension is added. If the extension does not match either a *.py, *rb, or *.cs, the Python configuration file is the default.

  • Graphics Module

The current graphics module has all the necessary objects to create graphic objects (points, circles, images, lines, etc). However, the current version is not threaded.

  • General Bug Fixes
    • Run-->Change Mode

In previous version of Pyjama, when pressing the R5 button, the code file is to be executed. However, suppose we have a ruby file open. The user would have to manually change the bottom console window to ruby to change the mode to run IronRuby (this too was not active in previous versions of Pyjama). In this version, when the user clicks R5, the top and bottom console window will change to the appropriate language and the file will be executed without errors (unless its a users error in the file).