• 0 Posts
  • 39 Comments
Joined 7 months ago
cake
Cake day: March 3rd, 2024

help-circle
  • All programs were developed in Python language (3.7.6). In addition, freely available Python libraries of NumPy (1.18.1) and Pandas (1.0.1) were used to manipulate data, cv2 (4.4.0) and matplotlib (3.1.3) were used to visualize, and scikit-learn (0.24.2) was used to implement RF. SqueezeNet and Grad-CAM were realized using the neural network library PyTorch (1.7.0). The DL network was trained and tested using a DL server mounted with an NVIDIA GeForce RTX 3090 GPU, 24 Intel Xeon CPUs, and 24 GB main memory

    it’s interesting that they’re using pretty modest hardware (i assume they mean 24 cores not CPUs) and fairly outdated dependencies. also having their dependencies listed out like this is pretty adorable. it has academic-out-of-touch-not-a-software-dev vibes. makes you wonder how much further a project like this could go with decent technical support. like, all these talented engineers are using 10k times the power to work on generalist models like GPT that struggle at these kinds of tasks, while promising that it would work someday and trivializing them as “downstream tasks”. i think there’s definitely still room in machine learning for expert models; sucks they struggle for proper support.






  • language is intrinsically tied to culture, history, and group identity, so any concept that is expressed through a certain linguistic system is inseparable from its cultural roots

    i feel like this is a big part of it. it reminds me of the Sapir Whorf Hypothesis. search results and neural networks are susceptible to bias just like a human is; “garbage in garbage out” as they say.

    the quote directly after mentions that newer or more precise searches produce more coherent results across languages. that reminds me of the time i got curious and looked up Marxism on Conservapedia. as you might expect, the high level descriptions of Marxism are highly critical and include a lot of bias, but interestingly once you dig down to concepts like historical materialism etc it gets harder to spin, since popular media narratives largely ignore those details and any “spin” would likely be blatant falsehood.

    the author of the article seems to really want there to be a malicious conspiratorial effort to suppress information, and, while that may be true in some cases, it just doesn’t seem feasible at scale. this is good to call out, but i don’t think these people who concern their lives with the research and advancement of language concepts are sleeping on the fact that bias exists.


  • it’s super weird that people think LLMs are so fundamentally different from neural networks, the underlying technology. neural network architectures are constantly improving, and LLMs are just a product of a ton of research and an emergence after the discovery of the transformer architecture. what LLMs have shown us is that we’re definitely on the right track using neural networks to solve a wide range of problems classified as “AI”














  • yeah i see that too. it seems like mostly a reactionary viewpoint. the reaction is understandable to a point since a lot of the “AI” features are half baked and forced on the user. to that point i don’t think GNOME etc should be scrambling to add copies of these features.

    what i would love to see is more engagement around additional pieces of software that are supplemental. for example, i would love if i could install a daemon that indexes my notes and allows me to do semantic search. or something similar with my images.

    the problems with AI features aren’t within the tech itself but in the surrounding politics. it’s become commonplace for “responsible” AI companies like OpenAI to not even produce papers around their tech (product announcement blogs that are vaguely scientific don’t count), much less source code, weights, and details on training data. and even when Meta releases their weights, they don’t specify their datasets. the rat race to see who can make a decent product with this amazing tech has made the whole industry a bunch of pearl clutching FOMO based tweakers. that sparks a comparison to blockchain, which is fair from the perspective of someone who hasn’t studied the tech or simply hasn’t seen a product that is relevant to them. but even those people will look at something fantastical like ChatGPT as if it’s pedestrian or unimpressive because when i asked it to write an implementation of the HTTP spec in the style of Fetty Wap it didn’t run perfectly the first time.