15-883 Homework #2
Computational Models of Neural Systems

Learning Goal

How to Run the CMAC Demo

Then cd to the directory CompNeuro/cmac1 When you're ready to begin, type "matlab" to start up Matlab. Then type "run" to start the demo.

Questions

1. Select the sine wave pattern (sin(x)), and click on the green line to add points. To get the MaxErr (the maximum error) below 0.2 as quickly as possible, what is your best strategy for selecting training points? Should you select points at random, or is there a more systematic approach? Justify your strategy and report the minimum number of points required.

2. Select the sin(3x) function and click the "x10" button to generate 10 random training points. Continue clicking "x10" until MaxErr is below 0.2. How many total random points did it take? Click "Reset" and repeat this experiment three more times to get a reliable average. Report the average number of points required.

3. Select the high-frequency sin(8x) function and generate at least 200 random points. Around what value does MaxErr seem to stabilize or "level out"? This value represents the model's approximate asymptotic error for this function with the current settings.

4. After MaxErr has stabilized for sin(8x) (as in the previous question), change the learning rate from 1.0 to 0.2 and continue training. Are you able to significantly reduce the error below the previous asymptotic value? Briefly explain why changing the learning rate has this effect.

5. Now, test the CMAC on the "Random" and "Step" functions. Qualitatively, do these functions seem easier or harder for the CMAC to learn compared to sin(3x)? Briefly explain your observations.

6. The hash_stride parameter controls the offset between hash tables, influencing how the model generalizes.
   1. Set the hash_stride to 1 and train on sin(8x).
   2. Reset, then set the hash_stride to 2, 3, 8, or 10 and train on sin(8x) again.
   3. For each setting, analyze how the stride affects:
      • Learning speed
      • The final asymptotic error.
      • The smoothness of the learned function.

   Based on this, explain the fundamental trade-off controlled by the stride size. What happens to generalization when the stride becomes very small or very large relative to the receptive field size of a unit?

7. Hash Tables vs. Buckets: The model's memory is determined by the number of hash tables (N Hashes) and the number of buckets in each (N Buckets/Hash). Experiment with at least three different combinations of these parameters while learning sin(8x). For example:
   1. High N Hashes, low N Buckets/Hash (e.g., 64 hashes, 5 buckets)
   2. Low N Hashes, high N Buckets/Hash (e.g., 16 hashes, 22 buckets)
   3. A balanced configuration of your choice.

   Train each configuration with 100 random points. Which configuration learns the high-frequency sin(8x) function most effectively? Why do you think the trade-off between more hash tables versus more buckets per table makes a difference?

8. Based on all your experiments, summarize in a few sentences the key characteristics of a function that make it difficult for a CMAC to learn effectively.