Signed in
We value your privacy
This site uses cookies to improve your browsing experience, analyze site traffic, and show personalized content.
Welcome to Pump.fun!
Pump lets anyone create coins, giving everyone equal access to buy and sell from the start. Prices can move quickly, so trade carefully.
By clicking this button, you agree to the Terms and Conditions, Privacy Policy, and certify that you are over 18 years old.
We value your privacy
This site uses cookies to improve your browsing experience, analyze site traffic, and show personalized content.
Swarming algorithms like PSO, Ant Colony, Sakana, and more in PyTorch 😊 # Swarms-Torch: Enterprise-Grade Swarm Intelligence Architectures [](https://discord.gg/jM3Z6M9uMq) [](https://www.youtube.com/@kyegomez3242) [](https://www.linkedin.com/in/kye-g-38759a207/) [](https://x.com/kyegomezb) --- ## Table of Contents - [Executive Summary](#executive-summary) - [Key Features](#key-features) - [Architecture Overview](#architecture-overview) - [Installation](#installation) - [Quick Start Guide](#quick-start-guide) - [Model Implementations](#model-implementations) - [Model Merging Techniques](#model-merging-techniques) - [Community & Support](#community--support) - [Documentation](#documentation) - [Contributing](#contributing) - [License](#license) - [Citations](#citations) --- ## Executive Summary **Swarms-Torch** is a cutting-edge PyTorch library that implements novel swarm intelligence architectures for next-generation AI systems. Our platform delivers 100% original swarming models designed to surpass traditional architectures like Transformers and State Space Models (SSMs). Built for enterprise-scale applications, Swarms-Torch provides production-ready implementations of bio-inspired algorithms including Particle Swarm Optimization with Transformers, Ant Colony systems, Neural Networks with Transformer synapses, and advanced Mixture of Experts architectures. --- ## Key Features ### 🔬 **Novel Architectures** - Particle Swarm Optimization with Transformer particles - Ant Colony Optimization with intelligent agents - Cellular Neural Networks with Transformer cells - Fish School/Sakana collective intelligence systems - Swarmalator dynamics simulation ### 🏗️ **Enterprise Components** - Mixture of Mambas with configurable fusion methods - Switch Mixture of Experts (SwitchMoE) - Simplified MoE implementations - Firefly optimization algorithms ### 🔧 **Advanced Model Merging** - HyperSlice merge techniques - Random subspace merging - Dimensional cross-fusion - Weighted evolutionary crossover - Permutation-based weight swapping ### 📈 **Production Ready** - Optimized for large-scale deployment - Comprehensive documentation - Extensive test coverage - Enterprise support available --- ## Architecture Overview Swarms-Torch implements bio-inspired collective intelligence patterns that leverage the emergent behaviors of natural swarms. Our architectures combine: - **Distributed Processing**: Multiple specialized agents working in parallel - **Emergent Intelligence**: Complex behaviors arising from simple interaction rules - **Adaptive Learning**: Dynamic optimization through collective feedback - **Scalable Design**: Efficient scaling from prototype to production --- ## Installation ### Requirements - Python 3.8+ - PyTorch 1.12+ - CUDA support recommended for optimal performance ### Install from PyPI ```bash pip install swarms-torch ``` ### Development Installation ```bash git clone https://github.com/kyegomez/swarms-pytorch.git cd swarms-pytorch pip install -e . ``` --- ## Quick Start Guide ### Basic Particle Swarm Optimization ```python from swarms_torch import ParticleSwarmOptimization # Initialize PSO with target optimization goal pso = ParticleSwarmOptimization( goal="Attention is all you need", n_particles=100 ) # Execute optimization process pso.optimize(iterations=1000) ``` ### Neural Network with Transformer Synapses ```python import torch from swarms_torch.nnt import NNTransformer # Create input tensor x = torch.randn(1, 10) # Initialize network architecture network = NNTransformer( neuron_count=5, num_states=10, input_dim=10, output_dim=10, nhead=2, ) # Forward pass output = network(x) ``` --- ## Model Implementations ### 1. Particle Swarm Optimization **Use Case**: Hyperparameter optimization, neural architecture search ```python from swarms_torch import ParticleSwarmOptimization pso = ParticleSwarmOptimization(goal="Attention is all you need", n_particles=100) pso.optimize(iterations=1000) ``` ### 2. Ant Colony Optimization **Use Case**: Combinatorial optimization, routing problems ```python from swarms_torch.ant_colony_swarm import AntColonyOptimization goal_string = "Hello ACO" aco = AntColonyOptimization(goal_string, num_iterations=1000) best_solution = aco.optimize() ``` ### 3. Cellular Swarm Networks **Use Case**: Distributed computing, parallel processing ```python from swarms_torch import CellularSwarm x = torch.randn(10, 32, 512) model = CellularSwarm(cell_count=5, input_dim=512, nhead=8) output = model(x) ``` ### 4. Fish School Intelligence **Use Case**: Collective decision making, ensemble learning ```python import torch from swarms_torch.fish_school import FishSchool src = torch.randn(10, 32, 512) tgt = torch.randn(10, 32, 512) labels = torch.randint(0, 512, (10, 32)) school = FishSchool(10, 512, 8, 6, 100) school.forward(src, tgt, labels) ``` ### 5. Mixture of Mambas **Use Case**: Large language models, sequence processing ```python import torch from swarms_torch import MixtureOfMambas x = torch.rand(1, 512, 512) model = MixtureOfMambas( num_mambas=2, dim=512, d_state=1024, depth=4, fusion_method="absmax" ) output = model(x) ``` ### 6. Switch Mixture of Experts **Use Case**: Sparse expert routing, efficient scaling ```python import torch from swarms_torch import SwitchMoE moe_layer = SwitchMoE( dim=768, hidden_dim=2048, output_dim=768, num_experts=16, use_aux_loss=False, ) x = torch.rand(32, 128, 768) output, auxiliary_loss = moe_layer(x) ``` ### 7. Firefly Optimization **Use Case**: Function optimization, genetic algorithms ```python from swarms_torch.firefly import FireflyOptimizer from torch import Tensor def rosenbrock(x: Tensor) -> Tensor: return (100 * (x[..., 1:] - x[..., :-1] ** 2) ** 2 + (1 - x[..., :-1]) ** 2).sum(dim=-1) optimizer = FireflyOptimizer(cost_function=rosenbrock) optimizer.optimize() best_solution = optimizer.get_best_solution() ``` --- ## Model Merging Techniques ### Advanced Fusion Methods ```python import torch from swarms_torch.mergers.all_new_evo_mergers import ( hyperslice_merge, random_subspace_merge, dimensional_cross_fusion, weighted_evolutionary_crossover, permutation_weight_swapping, ) # Initialize example models model_1 = torch.nn.Linear(10, 10) model_2 = torch.nn.Linear(10, 10) model_3 = torch.nn.Linear(10, 10) # HyperSlice merge merged_model_hs = hyperslice_merge( [model_1, model_2, model_3], slice_indices=[0, 2, 4] ) # Random Subspace merge merged_model_rs = random_subspace_merge( [model_1, model_2, model_3], subspace_fraction=0.5 ) # Weighted Evolutionary Crossover merged_model_wc = weighted_evolutionary_crossover( [model_1, model_2, model_3], performance_scores=[0.7, 0.85, 0.65] ) ``` --- ## Community & Support ### Official Resources | Resource | Description | Link | |----------|-------------|------| | **Documentation** | Comprehensive API documentation and tutorials | [swarmstorch.readthedocs.io](https://swarmstorch.readthedocs.io/en/latest/swarms/) | | **Discord Community** | Real-time support and discussions | [Join Discord](https://discord.gg/jM3Z6M9uMq) | | **Official Blog** | Latest updates and technical insights | [swarms.apac.ai](https://swarms.apac.ai) | | **Weekly Gatherings** | Community meetings every Thursday 1pm NYC | [Sign up here](https://lu.ma/5p2jnc2v) | ### Social Media & Updates | Platform | Purpose | Link | |----------|---------|------| | **Twitter/X** | Latest announcements and updates | [@swarms_corp](https://twitter.com/swarms_corp) | | **LinkedIn** | Professional network and company updates | [The Swarm Corporation](https://www.linkedin.com/company/the-swarm-corporation) | | **YouTube** | Video tutorials and demonstrations | [Swarms Channel](https://www.youtube.com/channel/UC9yXyitkbU_WSy7bd_41SqQ) | | **Personal Twitter** | Creator insights and development updates | [@kyegomezb](https://x.com/kyegomezb) | ### Getting Help | Type | Where to Go | Response Time | |------|-------------|---------------| | **Bug Reports** | [GitHub Issues](https://github.com/swarms/gateway/issues) | 24-48 hours | | **Feature Requests** | [GitHub Issues](https://github.com/swarms/gateway/issues) | 1-2 weeks | | **General Questions** | [Discord #help](https://discord.gg/jM3Z6M9uMq) | Real-time | | **Enterprise Support** | Contact via LinkedIn | 24 hours | --- ## Documentation - **[API Reference](https://swarmstorch.readthedocs.io/en/latest/swarms/)**: Complete documentation of all classes and methods - **[Examples](./playground/)**: Practical examples and implementation guides - **[Contributing Guide](./CONTRIBUTING.md)**: Guidelines for contributing to the project - **[Roadmap](https://github.com/users/kyegomez/projects/9)**: Development roadmap and future features --- ## Contributing We welcome contributions from the community! Swarms-Torch is an open-source project that thrives on collaboration. ### How to Contribute 1. **Pick an Issue**: Look for issues tagged with `good first issue` 2. **Fork the Repository**: Create your own fork of the project 3. **Make Changes**: Implement your feature or bug fix 4. **Submit PR**: Create a pull request with detailed description 5. **Review Process**: Collaborate with maintainers on feedback ### Areas of Contribution - **New Model Architectures**: Implement novel swarm intelligence patterns - **Performance Optimization**: Improve computational efficiency - **Documentation**: Enhance guides and API documentation - **Testing**: Expand test coverage and validation - **Bug Fixes**: Resolve existing issues **Read our full [Contributing Guidelines](./CONTRIBUTING.md)** ### Contributors <a href="https://github.com/kyegomez/swarms-pytorch/graphs/contributors"> <img src="https://contrib.rocks/image?repo=kyegomez/swarms-pytorch" /> </a> --- ## License This project is licensed under the MIT License. See the [LICENSE](./LICENSE) file for details. --- ## Citations If you use Swarms-Torch in your research, please cite: ### Firefly Algorithm ```bibtex @article{Yang2018WhyTF, title = {Why the Firefly Algorithm Works?}, author = {Xin-She Yang and Xingshi He}, journal = {ArXiv}, year = {2018}, volume = {abs/1806.01632}, url = {https://api.semanticscholar.org/CorpusID:46940737} } ``` ### Hybrid Genetic-Firefly Algorithm ```bibtex @article{article, author = {El-Shorbagy, M. and Elrefaey, Adel}, year = {2022}, month = {04}, pages = {706-730}, title = {A hybrid genetic-firefly algorithm for engineering design problems}, volume = {Journal of Computational Design and Engineering, Volume 9}, journal = {Journal of Computational Design and Engineering}, doi = {10.1093/jcde/qwac013} } ``` --- **© 2024 The Swarm Corporation. All rights reserved.** Source: https://github.com/kyegomez/swarms-pytorch
