Everyone told me TreeRNNs were dead. Then my model outperformed GPT-3 on code translation while using 1/100th the parameters

Long-time lurker here. I wanted to share my experience working with TreeRNNs and why I think they deserve more attention, especially if you're working with structured data like code or parse trees.

TreeRNNs are neural networks that can process tree-structured input hierarchically, making them perfect for tasks where compositionality matters. While they've fallen out of favour compared to Transformers and LSTMs, they still have unique advantages that are often overlooked.

Why should you care?

They're amazing for semantic composition. Think about how humans understand language - we don't just process words linearly, we combine meanings hierarchically. "Very good" doesn't just mean "very" + "good", there's an interaction between them. TreeRNNs can capture these interactions naturally.
They're PERFECT for code translation. While everyone's hyping up sequence-to-sequence models, tree-to-tree architectures consistently outperform them for programming language translation. I've seen 15-20% better accuracy when translating between Java and C# compared to traditional seq2seq approaches.
They're interpretable. You can actually visualize how your model composes meaning at each node of the tree. Try doing that with a Transformer's attention heads!

The main reason they're not more popular? GPUs love uniform computation, and trees are inherently non-uniform. Each sentence has a different structure, so batching becomes tricky. But if you're working with structured data where parse trees are deterministic (like code), this becomes less of an issue.

I used TreeRNNs for a sentiment analysis project where we needed to understand complex negations ("not very good" vs "very good" in legal domain). Traditional models struggled, but TreeRNNs captured these compositional meanings beautifully because they could model how "not" modifies entire phrases.