Lines of Questioning

• Scale-aware neural analysis. Looking at single-neuro-circuits, subsections, whole brain and cortexes, etc.
• How does sparsity affect our predictions? There is a difference between not having past experience in a domain and not being able to use that as predictive information, and having things you should now, but you forgot about it or suffered from amnesia, etc.
  • Sparse neural activations, often overlooked or nullified, may encode important contextual information that modulates surrounding neural activity and significantly impacts cognitive function and predictive capabilities. We need to be in high-dimensional space to identify this.

    • Do biological neural networks show evidence of leveraging sparse activations?

      • Selective amnesia
      • https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5861508/: Amnesia and its effects on recall, etc.
      • Systematically-biased forecasts

        • Selective Memory Equilibrium: https://economics.mit.edu/sites/default/files/inline-files/Selective_Memory_Equilibrium-5.pdf

          • Extension:

          Seems pretty applicable to model hallucinations!

          Other Possible Extensions It would be relatively easy to extend our analysis to agents who “misremember” and access false memories as opposed to simply forgetting things that happened. A more substantive generalization would be from an agent who believes that outcomes are i.i.d. to an agent who believes that outcomes follow a Markov process. This would let us capture, the gambler’s fallacy (see Rabin and Vayanos [2010] and He [2022]) if an outcome is more memorable when it is different than the outcome in the previous period.33 Or it might be much easier for agents to recall whether an experience happened at all than whether it happened five or six times; we could capture this by using a memory function that is concave in the number of times an experience occurred. Another generalization would be to memory functions with recency bias, such as ms1 ,tpsτ , aτ , yτ q “ ms1 psτ , aτ , yτ qf pt ´ τ q where f is a decreasing function. As with associative memory, when the outcomes are exogenous, this bias only leads to slower learning, but when actions are endogenous, it can prevent the agent from locking on to the optimal action.

      For this, I imagine we care about identifying and understanding the effects of sparse activations. We analyze this in high-dimensional space and differentiate sparse points (the absence of data that should be there) from null data.

Maintaining Plasticity in Deep Continual Learning (Dohare et. al, 2024)

https://arxiv.org/abs/2306.13812

• Dealing with the question of generalization and overfitting.
• More than this, we can continuously train our models.
  • When we do this:
    • The model may struggle to remember past examples with a bias to what has been recently learned. Its memory worsens. This is called “catastrophic forgetting”
    • The model is more novelly shown to lose its ability to learn new examples (requiring more training for the marginal example). Here, we observe that the plasticity decreases.
• How continual learning works: Data is continuously gathered from the setting, and then the model is intermittently retrained (from scratch). I assume this training happens on the pre-existing data in addition to the newly collected data.
• Continual learning is good when we have certain circumstances of a changing data stream. The layout of a house stays the same, for example, but as the robot navigates the space, it learns to add to the layout.
• It can be solved with continual backpropagation.
  • Normally, we initialize connection weights with small random numbers.
  • But if we do continual training usually we only initialize once, but we can initiate with each retraining step. We only do this for a small fraction, a subspace. Therefore, it takes place at all time steps, the continual backpropogation.
• How the plasticity decline was identified:
  • “The only difference in the learning algorithm across time are the network weights”
  • Initially, they are small numbers taken randomly from the initial distribution.
  • However, the weights adapt to the first task and the initial distribution for the subsequent task is based on the outcomes of the first, but this means we lose a lot of ability to learn.
  • There are some unique properties of random initialization that make backpropogation elastic in the beginning.
  • Protecting new units from immediate reinitialization by using a maturity threshold. We could set old units to 0 but this hurts our memory, initializing the outgoing weights affects our plasticity.
• Observed findings with the loss of plasticity:
  • There’s a continual increase in the fraction of constant units.
  • What this means is that many parameters are set to 0 and they become dead and essentially unchanged with regularization.
  • Without continual backprop, the decrease in performance is because of the sharp increase in dead units.
  • Similar to the rank of the matrix (linearly independent dimensions), the effective rank looks at how each dimension influences the transformation of the matrix.
  • Main contribution: Contribution utility—hidden sum of utilities of all outgoing connections.
  • Continual backprop helps give non-degradation.

Modeling behaviorally relevant neural dynamics enabled by preferential subspace identification (Sani et. al)

• Main contribution: Preferential subspace identification while taking into account relevant behaviors (versus. ignoring them completely or having noisy analyses)
• Unlike PCA, linear discriminant analysis, etc. PSID allows us to look at “continuous behavioral measurements.”
• Compared to other methods that work for continuous data like reduced rank regression, partial least squares regression, targeted dimensionality reduction, canonical correlation. With PSID we have dynamism and we can look at temporality. These methods work by building linear projections of one signal onto another (i.e. finding the largest covariations (variance) between neural activity in cortical areas.
• PSID allows us to extract behaviourally relevant low-dimensional representations for neural activity, and it does this by allowing for continuity, allowing us to work with dynamism and accepting a broad range of behaviours.
  • Definition: Ordinal time series (OTS) where we have a sequence of time-indexed values and the sequence of these discrete observations.