AI::NeuralNet::Mesh - An optimized, accurate neural network Mesh.
AI::NeuralNet::Mesh - An optimized, accurate neural network Mesh.
my $net = new AI::NeuralNet::Mesh(2,2,1);
This is version 0.20, the first release of this module.
FEEDBACK - Feedback is good. Feeedback is very good. No, not for my ego. :) Please write and let me know how your are using the module and/or how you think it could be better! Flames/comments/critiques are all welcome and appreciated.
AI::NeuralNet::Mesh is an optimized, accurate neural network Mesh. It was designed with accruacy and speed in mind. This is a neural net simulator similar to AI::NeuralNet::BackProp, but with several important differences. The two APIs are the same, that of this module and ::BackProp, so any scripts that use ::BackProp, should be able to use this module without (almost) any changes in your code. (The only changes needed will be to change the ``use'' line and the ``new'' constructor line to use ::Mesh instead of ::BackProp.)
This is a complete, from-scratch re-write of the Perl module AI::NeuralNet::BackProp. It a method of learning similar to back propogation, yet with a few custom modifications, includeding a specialized output layer, as well as a better descent model for learning.
Almost all of the notes and description in AI::NeuralNet::BackProp apply to this module, yet the differences I will detail below. I also have included a complete working function refrence here, with the updates added.
In AI::NeuralNet::BackProp, the neurons would be connected like this:
output
/ \
O O
|\ /|
| \/ |
| /\ |
|/ \|
O O
\ /
input
In this module, I have made a couple of important changes to the connection map. Consider this digram (This has 2 layers, 2 nodes/layer, 1 output node):
data collector
^
|
O <-- OUTPUT LAYER
/ \
/ \
O O <-- LAYER 2
|\ /|
| \ / |
| / \ |
|/ \|
O O <-- LAYER 1
| |
^ ^
input array
The mesh model includes an extra output "layer" above the final layer specified in the constructor. If the constructor had specified 2 layers, 2 nodes/layer, and B<2> output nodes, then the mesh would look like this:
data collector
^ ^
| |
O O <-- OUTPUT LAYER
| |
| |
O O <-- LAYER 2
|\ /|
| \ / |
| / \ |
|/ \|
O O <-- LAYER 1
| |
^ ^
input array
As you can see, the mesh creator adds one node in the output layer for every node called for in the constructor. This adds an node for that output, allowing better accuracy in the network, whereas in AI::NeuralNet::BackProp the output nodes were not allowed to be weighted.
In this module I have included a more accurate form of ``learning'' for the mesh. This form preforms descent toward a local error minimum (0) on a directional delta, rather than the desired value for that node. This allows for better, and more accurate results with larger datasets. This module also uses a simpler recursion technique which, suprisingly, is more accurate than the original technique that I used in AI::NeuralNet::BackProp.
By way of accuracy example, the included example script ``examples/ex_dow.pl'', upon the third learning loop (using AI::NeuralNet::BackProp), would almost always report forgetfulness around 25.00000% (rounded to five decimals), whereas when running the same example and the same example code with AI::NeuralNet::Mesh and only one learning loop, it reports forgetfulness of less than 2.00227! Over twenty-two percent increase in accuracy on one script alone.
The learning is also speed up immensly. Whereas the above mentioned script often take up to a half hour or more on my systems to learn the example data with the old AI::NeuralNet::BackProp module, it now (with this module) takes less than forty seconds to learn the data set (one loop).
AI::NeuralNet::Mesh
object. The network will have $layers number of layers in it
and it will have $nodes number of nodes per layer.
There is an optional parameter of $outputs, which specifies the number of output neurons to provide. If $outputs is not specified, $outputs defaults to equal $size.
Before you can really do anything useful with your new neural network
object, you need to teach it some patterns. See the learn() method, below.
You can also specify strings as inputs and ouputs to learn, and they will be crunched automatically. Example:
$net->learn('corn', 'cob');
Note, the old method of calling crunch on the values still works just as well.
The first two arguments may be array refs (or now, strings), and they may be of different lengths.
Options should be written on hash form. There are three options:
inc => $learning_gradient
max => $maximum_iterations
error => $maximum_allowable_percentage_of_error
$learning_gradient is an optional value used to adjust the weights of the internal connections. If $learning_gradient is ommitted, it defaults to 0.10.
$maximum_iterations is the maximum numbers of iteration the loop should do. It defaults to 1024. Set it to 0 if you never want the loop to quit before the pattern is perfectly learned.
$maximum_allowable_percentage_of_error is the maximum allowable error to have. If
this is set, then learn() will return when the perecentage difference between the
actual results and desired results falls below $maximum_allowable_percentage_of_error.
If you do not include 'error', or $maximum_allowable_percentage_of_error is set to -1,
then learn() will not return until it gets an exact match for the desired result OR it
reaches $maximum_iterations.
learn() (learn_set() uses learn() internally)
and allows you to specify a set to learn, rather than individual patterns.
A dataset is an array refrence with at least two elements in the array,
each element being another array refrence (or now, a scalar string). For
each pattern to learn, you must specify an input array ref, and an ouput
array ref as the next element. Example:
my @set = (
# inputs outputs
[ 1,2,3,4 ], [ 1,3,5,6 ],
[ 0,2,5,6 ], [ 0,2,1,2 ]
);
Inputs and outputs in the dataset can also be strings.
See the paragraph on measuring forgetfulness, below. There are two learn_set()-specific option tags available:
flag => $flag
pattern => $row
If ``flag'' is set to some TRUE value, as in ``flag => 1'' in the hash of options, or if the option ``flag''
is not set, then it will return a percentage represting the amount of forgetfullness. Otherwise,
learn_set() will return an integer specifying the amount of forgetfulness when all the patterns
are learned.
If ``pattern'' is set, then learn_set() will use that pattern in the data set to measure forgetfulness by.
If ``pattern'' is omitted, it defaults to the first pattern in the set. Example:
my @set = (
[ 0,1,0,1 ], [ 0 ],
[ 0,0,1,0 ], [ 1 ],
[ 1,1,0,1 ], [ 2 ], # <---
[ 0,1,1,0 ], [ 3 ]
);
If you wish to measure forgetfulness as indicated by the line with the arrow, then you would pass 2 as the "pattern" option, as in "pattern => 2".
Now why the heck would anyone want to measure forgetfulness, you ask? Maybe you wonder how I even measure that. Well, it is not a vital value that you have to know. I just put in a ``forgetfulness measure'' one day because I thought it would be neat to know.
How the module measures forgetfulness is this: First, it learns all the patterns
in the set provided, then it will run the very first pattern (or whatever pattern
is specified by the ``row'' option) in the set after it has finished learning. It
will compare the run() output with the desired output as specified in the dataset.
In a perfect world, the two should match exactly. What we measure is how much that
they don't match, thus the amount of forgetfulness the network has.
Example (from examples/ex_dow.pl):
# Data from 1989 (as far as I know..this is taken from example data on BrainMaker)
my @data = (
# Mo CPI CPI-1 CPI-3 Oil Oil-1 Oil-3 Dow Dow-1 Dow-3 Dow Ave (output)
[ 1, 229, 220, 146, 20.0, 21.9, 19.5, 2645, 2652, 2597], [ 2647 ],
[ 2, 235, 226, 155, 19.8, 20.0, 18.3, 2633, 2645, 2585], [ 2637 ],
[ 3, 244, 235, 164, 19.6, 19.8, 18.1, 2627, 2633, 2579], [ 2630 ],
[ 4, 261, 244, 181, 19.6, 19.6, 18.1, 2611, 2627, 2563], [ 2620 ],
[ 5, 276, 261, 196, 19.5, 19.6, 18.0, 2630, 2611, 2582], [ 2638 ],
[ 6, 287, 276, 207, 19.5, 19.5, 18.0, 2637, 2630, 2589], [ 2635 ],
[ 7, 296, 287, 212, 19.3, 19.5, 17.8, 2640, 2637, 2592], [ 2641 ]
);
# Learn the set
my $f = $net->learn_set(\@data,
inc => 0.1,
max => 500,
);
# Print it
print "Forgetfullness: $f%";
This is a snippet from the example script examples/finance.pl, which demonstrates DOW average prediction for the next month. A more simple set defenition would be as such:
my @data = (
[ 0,1 ], [ 1 ],
[ 1,0 ], [ 0 ]
);
$net->learn_set(\@data);
Same effect as above, but not the same data (obviously).
run() automatically benchmarked itself. In ::Mesh run() does
not do this inorder to speed up learn() by as much as 10-20 seconds on small
data sets. Even larger speed increases are realized on larger data sets.
This method will apply the given array ref at the input layer of the neural network, and
it will return an array ref to the output of the network. run() will now automatically crunch()
a string given as an input.
Example Usage:
my $inputs = [ 1,1,0,1 ];
my $outputs = $net->run($inputs);
You can also do this with a string:
my $outputs = $net->run('cloudy, wind is 5 MPH NW');
See also run_uc() below.
$net->uncrunch($net->run($input_map_ref));
All that run_uc() does is that it automatically calls uncrunch() on the output, regardless
of whether the input was crunch() -ed or not.
range() enabled, ::Mesh does not.
In this module, range() is disabled. It is included as a function stub
to comply with the API established by AI::NeuralNet::BackProp. I have
included the full code to the two essential parts of range() in the module
file, though. If anyone feels up to it, they can attempt to get range()
working on their own. If you do get range working, please send me a copy! :-)
benchmarked() returns benchmark info for last run() call.
In ::Mesh it only will return info for the last learn() call. benchmarked()
is an alias for benchmark() so we don't break any scripts..
This returns a benchmark info string for the last learn() call.
It is easily printed as a string, as following:
print "Last learn() took ",$net->benchmark(),"\n";
NOTE: Debugging verbosity has been toned down somewhat from AI::NeuralNet::BackProp, but level 4 still prints the same amount of information as you were used too. The other levels, however, are mostly for really advanced use. Not much explanation in the other levels, but they are included for those of you that feel daring (or just plain bored.)
Level 0 ($level = 0) : Default, no debugging information printed. All printing is left to calling script.
Level 1 ($level = 1) : Displays the activity between nodes, prints what values were received and what they were weighted too.
Level 2 ($level = 2) : I don't think I included any level 2 debugs in this version.
Level 3 ($level = 3) : Just prints info from the lear() loop, in the form of ``got: X, wanted Y''
type of information.
Level 4 ($level = 4) : This level is the one I use most. It is only used during learning. It displays the current error (difference between actual outputs and the target outputs you asked for), as well as the current loop number and the benchmark time for the last learn cycle. Also printed are the actual outputs and the target outputs below the benchmark times.
Toggles debuging off when called with no arguments.
crunch() . Also saves any output ranges set with range() .
This uses a simple flat-file text storage format, and therefore the network files should be fairly portable.
This method will return undef if there was a problem with writing the file. If there is an
error, it will set the internal error message, which you can retrive with the error() method,
below.
If there were no errors, it will return a refrence to $net.
save() and completly restore the internal
state at the point it was save() was called at.
If the file doesn't exist, or if the file is of an invalid file type, then load() will
return undef. To determine what caused the error, use the error() method, beelow.
If there were no errors, it will return a refrence to $net.
join() ,
it prints the elements of $array_ref to STDIO, adding a newline (\n) after every $row_length_in_elements
number of elements has passed. Additionally, if you include a $high_state_character and a $low_state_character,
it will print the $high_state_character (can be more than one character) for every element that
has a true value, and the $low_state_character for every element that has a false value.
If you do not supply a $high_state_character, or the $high_state_character is a null or empty or
undefined string, it join_cols() will just print the numerical value of each element seperated
by a null character (\0). join_cols() defaults to the latter behaviour.
sprintf() and int() , Provides
better rounding than just calling int() on the float. Also used very heavily internally.
load() and save()
calls. This is designed to be used to generate unique maps sutible for passing to learn() and
run() directly. It returns an array ref.
The words are not duplicated internally. For example:
$net->crunch("How are you?");
Will probably return an array ref containing 1,2,3. A subsequent call of:
$net->crunch("How is Jane?");
Will probably return an array ref containing 1,4,5. Notice, the first element stayed the same. That is because it already stored the word ``How''. So, each word is stored only once internally and the returned array ref reflects that.
crunch() method, above, possibly to
uncrunch() the output of a run() call. Consider the below code (also in ./examples/ex1.pl):
use AI::NeuralNet::Mesh;
my $net = AI::NeuralNet::Mesh->new(2,3);
for (0..3) {
$net->learn_set([
$net->crunch("I love chips."), $net->crunch("That's Junk Food!")),
$net->crunch("I love apples."), $net->crunch("Good, Healthy Food.")),
$net->crunch("I love pop."), $net->crunch("That's Junk Food!")),
$net->crunch("I love oranges."),$net->crunch("Good, Healthy Food."))
]);
}
print $net->run_uc("I love corn.")),"\n";
On my system, this responds with, ``Good, Healthy Food.'' If you try to run crunch() with
``I love pop.'', though, you will probably get ``Food! apples. apples.'' (At least it returns
that on my system.) As you can see, the associations are not yet perfect, but it can make
for some interesting demos!
If the word is not in the list, it will set the internal error value with a text message
that you can retrive with the error() method, below.
It will return the current width when called with a 0 or undef value.
The column width is preserved across load() and save() calls.
load() and save() calls.
run() or learn() -ed, to prevent the
network from hanging on a 0 value. When called with no arguments, it returns the current
const. value. It defaults to 0.0001 on a newly-created network. The run const. value
is preserved across load() and save() calls.
error() method, below.
This is a treat... this routine will load a PCX-format file (yah, I know ... ancient format ... but it is the only one I could find specs for to write it in Perl. If anyone can get specs for any other formats, or could write a loader for them, I would be very grateful!) Anyways, a PCX-format file that is exactly 320x200 with 8 bits per pixel, with pure Perl. It returns a blessed refrence to a PCX::Loader object, which supports the following routinges/members. See example files ex_pcx.pl and ex_pcxl.pl in the ./examples/ directory.
The methods below are basically the same as what you would find when you type:
% perldoc PCX::Loader
$pcx->{palette}->[0]->{red};
$pcx->{palette}->[0]->{green};
$pcx->{palette}->[0]->{blue};
Each is in the range of 0..63, corresponding to their named color component.
[$left,$top,$right,$bottom]
These must be in the range of 0..319 for $left and $right, and the range of 0..199 for $top and $bottom. The block is returned as an array ref with horizontal lines in sequental order. I.e. to get a pixel from [2,5] in the block, and $left-$right was 20, then the element in the array ref containing the contents of coordinates [2,5] would be found by [5*20+2] ($y*$width+$x).
print $pcx->get_block(0,0,20,50)->[5*20+2];
This would print the contents of the element at block coords [2,5].
Rodin Porrata asked on the ai-neuralnet-backprop malining list, ``What can they [Neural Networks] do?''. In regards to that questioin, consider the following:
Neural Nets are formed by simulated neurons connected together much the same way the brain's neurons are, neural networks are able to associate and generalize without rules. They have solved problems in pattern recognition, robotics, speech processing, financial predicting and signal processing, to name a few.
One of the first impressive neural networks was NetTalk, which read in ASCII text and correctly pronounced the words (producing phonemes which drove a speech chip), even those it had never seen before. Designed by John Hopkins biophysicist Terry Sejnowski and Charles Rosenberg of Princeton in 1986, this application made the Backprogagation training algorithm famous. Using the same paradigm, a neural network has been trained to classify sonar returns from an undersea mine and rock. This classifier, designed by Sejnowski and R. Paul Gorman, performed better than a nearest-neighbor classifier.
The kinds of problems best solved by neural networks are those that people are good at such as association, evaluation and pattern recognition. Problems that are difficult to compute and do not require perfect answers, just very good answers, are also best done with neural networks. A quick, very good response is often more desirable than a more accurate answer which takes longer to compute. This is especially true in robotics or industrial controller applications. Predictions of behavior and general analysis of data are also affairs for neural networks. In the financial arena, consumer loan analysis and financial forecasting make good applications. New network designers are working on weather forecasts by neural networks (Myself included). Currently, doctors are developing medical neural networks as an aid in diagnosis. Attorneys and insurance companies are also working on neural networks to help estimate the value of claims.
Neural networks are poor at precise calculations and serial processing. They are also unable to predict or recognize anything that does not inherently contain some sort of pattern. For example, they cannot predict the lottery, since this is a random process. It is unlikely that a neural network could be built which has the capacity to think as well as a person does for two reasons. Neural networks are terrible at deduction, or logical thinking and the human brain is just too complex to completely simulate. Also, some problems are too difficult for present technology. Real vision, for example, is a long way off.
In short, Neural Networks are poor at precise calculations, but good at association, evaluation, and pattern recognition.
Included are several example files in the ``examples'' directory from the distribution ZIP file.
ex_dow.pl
ex_add.pl
ex_add2.pl
ex_pcx.pl
ex_pcx2.pl
ex_alpha.pl
ex_bmp.pl
ex_bmp2.pl
ex_letters.pl
Each of these includes a short explanation at the top of the file. Each of these are ment to demonstrate simple, yet practical uses of this module.
These packages are not designed to be called directly, they are for internal use. They are listed here simply for your refrence.
This is the beta release of AI::NeuralNet::Mesh, and that holding true, I am sure
there are probably bugs in here which I just have not found yet. If you find bugs in this module, I would
appreciate it greatly if you could report them to me at <jdb@wcoil.com>,
or, even better, try to patch them yourself and figure out why the bug is being buggy, and
send me the patched code, again at <jdb@wcoil.com>.
Josiah Bryan <jdb@wcoil.com>
Copyright (c) 2000 Josiah Bryan. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
The AI::NeuralNet::Mesh and related modules are free software. THEY COME WITHOUT WARRANTY OF ANY KIND.
FEEDBACK - Feedback is good. Feeedback is very good. No, not for my ego. :) Please write and let me know how your are using the module and/or how you think it could be better! Flames/comments/critiques are all welcome and appreciated.
$Id: AI::NeuralNet::Mesh.pm, v0.20 2000/23/12 05:05:27 josiah Exp $
You can always download the latest copy of AI::NeuralNet::Mesh from http://www.josiah.countystart.com/modules/get.pl?mesh:mesh.htm
A mailing list has been setup for AI::NeuralNet::BackProp. I am going to use that list to announce and discuss this module, AI::NeuralNet::Mesh. The list is for discussion of AI and neural net related topics as they pertain to AI::NeuralNet::BackProp and AI::NeuralNet::mesh. I will also announce in the group each time a new release of AI::NeuralNet::BackProp is available.
The list address is: ai-neuralnet-backprop@egroups.com