	Reinforcement Learning and Artificial Intelligence (RLAI)
	Tile Coding Software

The ambition of this page is to very briefly describe the tile coding functions and their various versions and implementations such that readers can quickly determine whether or not they are likely to meet their needs. External documentation is not here but is pointed to from here.

Tile coding is a way of representing the values of a vector of continuous variables as a large binary vector with few 1s and many 0s. The main step is to partition, or tile, the continuous space multiple times and select one tile from each tiling, that corresponding the the vector's value. Each tile is converted (hashed down to) an element in the big binary vector, and the list of the tile (element) numbers is returned as the representation of the vector's value. Information and examples of tile coding and its use in function approximation and reinforcement learning are given in Section 8.3.2 of the RL textbook. The version provided here includes code for handling collisions and a wrap version as well.

Note: the new version (as of January 25, 2005) has different function names than previous versions. The function getTiles is now tiles, loadTiles is loadtiles, tilesWrap is tileswrap and loadTilesWrap is now loadtileswrap. The arguments to these functions have not changed, and neither has CollisionTable.

Reference manual
Python Tiles

Download and installation instructions

Python calling C version of Tiles

Download and installation instructions

Tiles for C++

Download and installation instructions

Tiles for Macintosh Common Lisp

Download and installation instructions

Comparison Timings

Python Installation of Tiles

Requirements:

You must have the following installed on your machine to use tiles:
- Python (version 2.3)
For help in using Python, see here.

Installation:

The Tiles code was last updated January 25, 2005. For previous versions, see information on RLtoolkit versions.
Download python source code here. Right click or control click to save this tarball.
The file will unzip into a folder called Tiles. Move this folder to your Python site packages folder, or to wherever you want to run Python from.
Tiles can also be acquired by installing the Reinforcement Learning toolkit.

Usage:

Move to the folder you put Tiles in (not necessary if it is installed in your python site packages folder)
Start up Python and do one of the following:
- from Tiles.tiles import *
  (then you can call the basic tiles functions tiles, loadtiles, tileswrap, loadtileswrap and CollisionTable with no module name required)
- import Tiles.tiles as tiles
  (then make sure you prefix all basic tiles commands with tiles - eg. tiles.loadtiles(...)
- from Tiles import *
  Then you can use the various modules in the Tiles package as follows by prefixing the module name to the desired function. For example:
  - tiles.tiles(...)
  - fancytiles.diamondtiles(...)
  - tilesdemo.showtiles(...)

Python Calling C Version of Tiles: Installation

This version allows you to have the extra speed of the C version of tiles but still write your programs in Python.

Requirements:

You must have the following installed on your machine to use the c version of tiles:
- Python (version 2.3)
- C++
For help in using Python, see here.

Installation:

The Python calling C Tiles code was last updated January 25, 2005.
Download C and python source code here. Right click or control click to save this tarball.
The file will unzip into a folder called CTiles. Move this folder to your Python site packages, or to wherever you want to run Python from.
There is a Makefile for Macintosh in this folder. You must run this to compile the c version of tiles, and the python to c interface for tiles. If you are not using a Macintosh, look at the Makefile to see what needs to be done and compile the code appropriately for your platform.
Tiles, including this version, can also be acquired by installing the Reinforcement Learning toolkit.

Usage:

Move to the folder you put CTiles in (not necessary if it is installed in your python site packages folder)
Start up Python and do one of the following:
- from CTiles.tiles import *
  (then you can call the basic tiles functions tiles, loadtiles, tileswrap, loadtileswrap and CollisionTable with no module name required)
- import CTiles.tiles as tiles
  (then make sure you prefix all basic tiles commands with tiles - eg. tiles.loadtiles(...)
- from CTiles import *
  Then you can use the various modules in the CTiles package as follows by prefixing the module name to the desired function. For example:
  - tiles.tiles(...)
  - fancytiles.diamondtiles(...)
  - tilesdemo.showtiles(...)

Lisp Installation of Tiles

Requirements:

This runs under Macintosh Common Lisp

Installation:

The tiles.lisp source code is available here. Right click or control click to save this file. Timing code is here.
(Previous version is here)

C Installation of Tiles

Requirements:

You must have the following installed on your machine to use tiles:
- C++

Installation:

C++ header, C++ source Timing code is here
(Previous version: C++ header, C++ source)

Timing Results

For comparison timings, see the following table. The tests were run on a Mac powerbook with 1 G memory. For 10,000 calls to tiles, each with 4 tilings and 2 floats:

Tiles Version	Python	Lisp	C	Python calling C
Regular: no collision table	1.03 sec	0.19 sec	0.02 sec	0.09 sec
unsafe collision table	1.56 sec	0.26 sec	0.06 sec	0.10 sec
safe collision table	1.73 sec	0.29 sec	0.04 sec	0.10 sec
super safe collision table	2.19 sec	0.24 sec	0.06 sec	0.10 sec
Wrap: no collision table	1.50 sec	1.38 sec(GC)	0.02 sec	0.12 sec
unsafe collision table	1.86 sec	0.29 sec	0.05 sec	0.11 sec
safe collision table	1.93 sec	0.29 sec	0.03 sec	0.13 sec
super safe collision table	1.64 sec	0.25 sec	0.05 sec	0.12 sec
Load: no collision table	1.22 sec	0.27 sec	0.0003 sec	5.9 sec
unsafe collision table	1.81 sec	1.57 sec(GC)	0.0004 sec	5.8 sec
safe collision table	1.96 sec	0.31 sec	0.0004 sec	5.9 sec
super safe collision table	1.78 sec	0.27 sec	0.0004 sec	6.4 sec
Load wrap: no collision table	1.39 sec	0.21 sec	0.0002 sec	5.8 sec
unsafe collision table	1.87 sec	0.30 sec	0.0003 sec	5.9 sec
safe collision table	1.95 sec	0.29 sec	0.0003 sec	5.8 sec
super safe collision table	1.62 sec	0.35 sec	0.0005 sec	5.9 sec

Extend this Page How to edit Style Subscribe Notify Suggest Help This open web page hosted at the University of Alberta. Terms of use 3668/1