[Nov 15th Event] It Ain't Broke So D̶o̶n̶'t̶ F̶i̶x̶ Let's Break It

Use this topic to ask your questions to Jakob Uszkoreit during his talk: It Ain’t Broke So D̶o̶n̶’t̶ F̶i̶x̶ Let’s Break It.

You can watch it on YouTube or on Twitch at 11:45am PST.

What does it mean to “marginalize out” dependencies?

My question is about transformers limitations : Given several codes written in different programming languages (java, python, C ++ etc). Suppose we have a huge dataset of these codes and we want to determine similar and not similar codes(similar = solving the same problem). Here The Transofmers (by their architecture) have limitations to solve this probelm because it is not about statistical understanding of the lanaguge. here it is about detecting similar structures between different programming languages (and not human language). Do you think that GNN (Graphical Neural Networks) could be a solution in this case, otherwise what is your proposal to tackle this problem ?

My question is about transformers limitations : Given several codes written in different programming languages (java, python, C ++ etc). Suppose we have a huge dataset of these codes and we want to determine similar and not similar codes(similar = solving the same problem). Here The Transofmers (by their architecture) have limitations to solve this probelm because it is not about statistical understanding of the lanaguge. here it is about detecting similar structures between different programming languages (and not human language). Do you think that GNN (Graphical Neural Networks) could be a solution in this case, otherwise what is your proposal to tackle this problem ?

This is answered at 4:32:30 in the main stream.

Does anyone know the Nov 3 paper Jakob was referring to?

With regards to the talk of Jakob Uszkoreit, I am wondering whether
symbolic languages like Chinese, which have originated from abstract image
representations of objects might be more likely to generalize in cross-modality joint modeling approaches of image and text, if represented appropriately. In the end also the compositional hierarchy of Chinese ideogram symbols (composed of multiple Chinese characters as radicals to derive a novel character with the ideas from those original characters combined). Since language is likely to have somewhat originated from cave drawings, and those simplified image representations are more closely related to 3D objects in the real world it might be possible to represent languages like Chinese more easily. Maybe by jointly utilizing images of the Chinese characters + token sequence representations of Chinese characters and the real-world image of the object.