This revision is from 2024/08/02 06:33. You can Restore it.
A.I. computing has extreme next-generation computational demands. GPUs now dominate A.I. processing but remain confined to a card in a slot. Future designs may see the graphics card become the motherboard, absorbing the current paradigm. Computing advances focus on maximizing speed, getting components as close to the speed of light as possible, and leveraging parallelism by using thousands of simplified processors on a die for AI-specific operations like matrix calculations and hashing. Technologies such as CUDA cores, Tensor Processing Units (TPUs), and stream processors aim to optimize AI workloads through massive parallel processing.
GPU: Most important. Cards like RTX 3060 or RX 6600 significantly improve training speed, Nvidia GeForce RTX 4090 with 16,384 CUDA cores and 24GB GDDR6X VRAM. Adding multiple cards and interconnecting them using NVLink. Kubernetes clusters, infiniband to increase capacity even more. PCIv3 to NVLink and PCIv7. High-bandwidth interconnect for GPU-to-GPU communication. NVidia is kicking ass in this field.
CPU: Less critical. Mid-range processors with good balance of core count and single-core performance suffice, such as AMD Ryzen 5/7 or Intel Core i5/i7. In 2024, consumer CPUs typically range from 6 to 32 cores.
RAM: Moderately important. 32GB to 64GB is usually sufficient for most AI tasks. RAM disks can be utilized for faster data access.
Storage: Least critical. SATA SSDs are sufficient for most tasks, with NVMe SSDs offering faster data access when needed.
Training LLM's is another story.
Power consumption, utilizing renewable energy, solar, wind, maximal location.
Distributed computing software and clutering, MPI (Message Passing Interface). Using software such as TensorFlow Distributed, Spark or Cluster management software like Slurm or Torque. Petals, Horovod, a distributed training framework for libraries like TensorFlow, Keras, PyTorch, and Apache MXNet. Something like Kuberetes.
ASUS Pro WS WRX80E-SAGE SE WIFI II 2TB ram 8 16x PCIe, $1700, specs
Asrock WRX90 WS EVO - 2TB ram 7 x 16x PCIe + 1 8x, $1750, specs
Gigabyte MZ73-LM0 - DDR5 x 16, PCIe v4 $2550, specs
RAM: RDIMM, LRDIMM or UDIMM, 16x 128GB 3DS LRDIMM modules, total of 2TB RAM. Modules operate at 2400MHz
Storage: Any
PSU: Biggest, make sure quantity of connectors and compatible connectors, enough power and connectors for multiple GPUs. Corsair AX1600i (1600W, sufficient for multiple high-power GPUs).
EATX Case, just screw it down to something and earth it
Cooling: Custom liquid cooling loop to maintain optimal temperatures, to manage the heat output of multiple GPUs.
Riser from Supermicro: https://www.supermicro.com/en/support/resources/riser and take care that the included Molex to ATX cable could fry the riser, EVGA cable has a different pin power arrangement x. PCI-E 16X Riser Card Flexible Ribbon Extension Cable for space.
Multi-build clustering: hook them up in a conventional network and then utilize Distributed Computing Framework, install a chosen framework on each computer and configure it to recognize the other machines as part of the cluster. Allocate 1 machine as a NAS, a mobo with the most PCIe SATA expansion cards and onboard SATA. The other computers are about cpu, gpu cores and maxmimum vRAM.
When selecting graphics cards for running large language models (LLMs) locally, using multiple GPUs, there are several important features and specifications to consider:
High VRAM: Aim for graphics cards with as much VRAM as possible. Since you're looking to run LLMs, more VRAM allows you to handle larger models and batch sizes.
CUDA Cores / Tensor Cores: More CUDA cores generally mean better parallel processing capabilities. Tensor cores (found in NVIDIA’s RTX and Tesla series) are specifically designed for deep learning tasks and can significantly speed up model training and inference.
NVLink Support: NVLink allows for high-bandwidth communication between GPUs, enabling efficient multi-GPU setups. This is crucial for model parallelism and reducing inter-GPU communication overhead.
Multi-GPU Scalability: Ensure the graphics card and your system support multi-GPU configurations (e.g., via SLI, NVLink, or PCIe slots).
FP16 / Mixed Precision Support: Cards that support FP16 or mixed precision calculations can provide significant performance boosts for deep learning tasks by using less memory and speeding up computations.
Cooling System: Efficient cooling is essential to maintain performance and prevent thermal throttling, especially in multi-GPU setups.
Driver and Software Support: Ensure the card is compatible with the deep learning frameworks you plan to use (e.g., PyTorch, TensorFlow) and that it has robust driver support.
Recommended GPU Models
NVIDIA RTX 30 Series (e.g., RTX 3090, RTX 3080):
High VRAM (e.g., 24GB on RTX 3090)
Tensor cores for deep learning
NVLink support (for RTX 3090)
NVIDIA A100:
Up to 80GB VRAM (in the PCIe version)
Advanced tensor cores
NVLink support
Designed specifically for AI workloads
NVIDIA Tesla V100:
Up to 32GB VRAM
Tensor cores
NVLink support
NVIDIA Quadro RTX 8000:
48GB VRAM
Tensor cores
NVLink support
In non-SXM2 form factors, specifically the PCIe form factor for NVIDIA GPUs, the card has a typical PCIe edge connector that slides into the PCIe slot on the motherboard (connector is the edge of the pcb that slides into the motherboard slot). For PCIe form factor GPUs to support NVLink, there is an additional edge connector located towards the top of the card. This additional connector is used to attach an NVLink or SLI bridge, which allows for high-speed communication between multiple GPUs.
Note: some of these models may have different memory configurations depending on the specific model or revision. FP16 performance is not available on older cards like the K-series due to their lack of Tensor Cores. The Tesla T4 has Tensor Cores, which are designed to accelerate deep learning operations.
BIOS Settings: Ensure the BIOS is configured to support multi-GPU setups.
Driver Installation: Install the latest NVIDIA drivers that support multi-GPU configurations.
Framework Configuration: In your deep learning framework, configure the settings to utilize multiple GPUs (e.g., using torch.nn.DataParallel or torch.distributed in PyTorch).
Summary
By focusing on high VRAM, CUDA/Tensor cores, NVLink support, and efficient cooling, you can build a powerful multi-GPU setup capable of running large language models locally. Using high-end GPUs like the NVIDIA RTX 3090 or the A100 will provide the performance needed for demanding AI tasks.
Software
Software has become secondary to hardware, and software for A.I. would probably require grid computing in exchange for unrestricted model access. Each node would have to satisfy minimum requirements to be accepted into the grid. While the models are accessible to the grid, the secret source is with the author. The grid acts as a workshop, holding the petabytes of training data, and an A.I. training supercomputer. The result is plopped into the distributed leaderboard folder, where all the trained models go, and all the models are restricted to the OS, all the models are graded. A general user would go to the leaderboard folder and run the latest models. The incentive is to beat the best model. In the modern day, it is all about creating the white paper and presenting it to key people for support and funding. In the past, it anyone could release and gain public support organically.
O.I. Architecture - organoid on chip support
Issues:
not big enough, intelligence is relative to quantity of neurons in a network.
questionable unnatural systems and understanding natural habitats, where movement, re-organization, synapses, vascularization occur and are supported.
questions over lifespan, understanding their lifespan in the body versus their lifespan as bioware.
Building the network...
modularization and module versioning.
interconnects between organoids that mimic synapses, attraction, affinity.
communication, I/O cards, hardware and software interface. Understanding and responding.
a happiness rating.
distributed networking over distance (for fun).
nb: Organoids are real lifeforms.
Automating organoid maintenance
A pump, a reservoir and an input output system attachment on the container housing the organoid. The pump (heart) moves media from a reservoir into the input of the housing of the organoid and at the output move goes back to the pump so the media is circulating. Spent media gets moved to a storage container where it is measured, filtered and conditioned and then re-introduced. So 4 main objects are required. A slow pump, pipes and fittings from the pump to the organoid housing. A reservoir holding new media, and a container holding old media. In the old media container, additions such as media grading detection, filtration and media conditioning to recycle media. Like a filter in a fish tank, this slow moving pump keep the water clean and oxygenated, removing impurities.
a peristaltic pump, also commonly known as a roller pump, is a type of positive displacement pump used for pumping a variety of fluids. The fluid is contained in a flexible tube fitted inside a circular pump casing. Most peristaltic pumps work through rotary motion, though linear peristaltic pumps have also been made.
culture media filtration, sterilization as it passes through the filter (immune system) and a measure of it viability, supplementing and cleaning the media or impurities.
Ideally, we want to award these functions their organ names, use and develop human compatible artificial organs and machines to offshoot into the medical device industry. For example, the dialysis machine could be supplied to hospitals and work in that setting. Every time we do something, we must think of its application in general medicine and move towards that direction, even if it poses extra challenges. For example, anastomosis methods and materials and degree of identical behavior. Bioreactors are commonly used for cell culture applications.
Neuroplatform model, recent advacement offers an online interface online similar to a task sceduling print pool jobs for the organoids, results are returned for processing. Other similar bigger platforms might be integrated in future.
GPU to GPU communication, cluster and distributing networking training and testing. The amount of data and processing required to train models and tinker about with A.I. could utilize grid computing. Minimum requirements are mandatory to join the grid, and trained models are the reward. The grid would hold the petabytes of training data and CPU cycles for distributed training. A member would probably need some minimum resource to join the grid. The models are tied to the OS and cannot be moved out. The grid maintainers would keep models at the current or exceeding current capability. The use of the models to generate video, images and so on would be unrestricted. Make a trade of model access for resource contribution.
Installed with all the applications/software in the category and to leverage A.I. and O.I.
Master simulation environment for A.I. training.
Store training data - distributed file systems, to grid and store the petabytes of training data.
Train A.I. - utilize the many grid computing operations already in existance and add a system level one as well.
Neuron Communication: Neurons communicate with each other through electrical and chemical signals. The "language" they use is a complex interplay of these signals, which can be described in terms of neurotransmitters, receptors, and action potentials. Action Potentials: Neurons generate electrical signals called action potentials, which are rapid, temporary changes in the electrical potential difference across the neuron's membrane. These signals travel down the neuron's axon to its terminals. Neurotransmitters: When an action potential reaches the end of a neuron's axon, it triggers the release of neurotransmitters, which are chemical messengers stored in tiny sacs called vesicles. There are many types of neurotransmitters, such as glutamate, GABA, acetylcholine, serotonin, and dopamine, each with different functions. Receptors: Neurotransmitters bind to specific receptor proteins on the membrane of the target neuron, causing changes in the electrical potential of that neuron. This can either excite the neuron, making it more likely to fire an action potential, or inhibit it, making it less likely to fire. Synaptic Transmission: The process of neurotransmitter release, binding to receptors, and the resulting changes in the target neuron is called synaptic transmission. This is the fundamental way neurons communicate with each other. The complex system of communication involves electrical and chemical signals. This system allows them to transmit, process, and store information, forming the basis for all brain functions.
Neuromorphics
Neuromorphic processing unit, or NPU, designed to replicate features of the human brain. Intel's Loihi 2 processor. Analog chip or Digital chip. TrueNorth IBM's Neuromorphic Chip.
IMMORTALITY 
