We all know what Generative AI (GenAI) is and how it has become a transformative force in our society by revolutionizing various aspects of our lives. According to the books, GenAI is a subset of deep learning, which is a machine learning (ML) technique, and ML is a branch of larger artificial intelligence (AI) construct that enables machines to gain knowledge and enhance their performance through experience, without the need for direct programming. If this sounds confusing, then simply put, GenAI creates original content in response to user prompts. It can generate text, images, videos, music or even software code without human intervention. The sheer number of applications from generating digital masterpieces, composing symphonies or drafting captivating stories to a health diagnosis, being a virtual assistant, improving gaming experience and early fraud detection —all without human guidance sets this apart from all the emerging technologies over the last few decades. And clearly, the numbers do not lie.

The expected growth trajectory and market size of GenAI is nothing short of remarkable

• According to McKinsey, Gen AI features could add up to $ 4.4 trillion annually to the global economy and increase the impact of all artificial intelligence by 15–40%.
• The market size for generative AI is projected to reach $36.06 billion by 2024, with an impressive annual growth rate of 46.47% from 2024 to 2030.
• Statista estimates that by 2030, the market could soar to a staggering $356.10 billion.

Some common use cases that are fueling this GenAI growth can be largely categorized by

• Customer service enhancement across verticals like retail, travel, hospitality and ecommerce.
• Implementation of virtual assistant solutions such as patient assistance, financial advisory, smart service desk/ remote support, etc.
• Employee empowerment to streamline and automate tasks like project management, code creation, data analysis for business decisions and SOP (Standard Operating Procedures) recommendations.
• Generating innovative content and assets like music, art and texts. Creating marketing material in the ecommerce space, creating visual assets, marketing campaigns, etc.
• Predictive capabilities like fraud detection, predictive maintenance, medical diagnosis, audit compliance, etc.
• Optimization in workflows or processes with simulations and adaptive learning like financial advisory, construction modeling, manufacturing process simulations, network performance tuning, route planning, drug discovery etc.

GenAI and IT infrastructure

GenAI can not only perform your mundane tasks but also help you with creative thinking and ideation. Let’s try to see this in the context of IT Infrastructure.

• It can create optimization strategies for cloud or data center operations while AI dynamically optimizes the allocation of resources based on real-time demand and historical patterns. It allocates resources to critical workloads and efficiently manages non-essential tasks; performance is optimized and costs are reduced. This agility ensures better overall efficiency.
• GenAI can be deployed to simulate different scenarios playing with space, power and cooling requirements in a data center. This can help to optimize the IT density and add additional IT load while improving DC sustainability.
• GenAI can help improve cybersecurity by creating test cases. Along with predictive analysis it can help detect vulnerabilities, do phishing detection, perform automated security patching schedules, enhance biometrics and create threat simulations.
• GenAI can also help to enhance networking by writing programs and scripts, creating documentation, formulating policies and configurations, assisting with audits, responding to incidents, act as a virtual assistant or mentor to network professionals. Additional use cases include automated ticketing in data centers, IT issue troubleshooting, personalized support chatbots and remote support.

Well, this looks all good and great, but the true story in the context of IT infrastructure is rather a mixed bag. Within the IT infrastructure domain or elsewhere, a lot of computing power is needed to cater to the demands of all these use cases.

The impact of Generative AI on IT infrastructure

Let’s consider all 3 sources of IT infrastructure that can power GenAI workloads, i.e., Cloud providers, Colocation providers and Enterprise Datacenters to understand the impact of the increase in GenAI demand or IT infra demand at a much more granular level. As publicly accessible generative AI models like Midjourney and ChatGPT gain popularity, interest in their economic potential has translated into a tsunami of AI demand. This surge in demand is driving a frenzy of data center leasing, with approximately 2.1GW of data center leases signed in the U.S. between mid-May and late July. The cloud providers or Hyperscalers are turning to colocation operators for capacity needs. We must acknowledge that AI data center architectures are still relatively unexplored territory and improvements will come in to optimize the same, but even taking that into account, about 7% of the global power production will be consumed by DC and comms infra by 2030. This not only puts a question on the implausible use of IT resources but also on the use of GenAI itself. Those working in the GenAI space must also consider the impact this technology is having on our environment and understand if the benefits of GenAI outweigh the cost in a true sense.

As we are all aware, due to the novel nature of GenAI, there is a barrier to hardware and GPU (Graphics Processing Unit) manufacturers are facing bottlenecks due to limited chip supply. This is creating a monopolistic situation where the GPU manufacturers are kingmakers and Hyperscalers are kings. The cloud route seems quite lucrative, given the lower upfront costs. But as we go deeper into this cloud vs on-prem conversation, it turns out that the more cost-effective alternative will depend on where you are on your cloud journey. Is your data on-prem, or already on the cloud? How intensively will you be building your AI capabilities? and what is the end goal that you are trying to achieve? We must understand that the overall TCO (Total Cost of Ownership) or the cost of running GenAI workloads on-premise might be lower than that on the cloud, provided you have already tested the waters with the cloud and have figured out what is your specific requirement in terms of hardware and AI strategy.

Secondly, other than the hardware cost, the advent of private AI where private data, sensitive data like employee information or company financials are used as data sets while training the AI model along with other public LLMs (Large Language Models) is also sparking the enterprise DC conversation.

The next unique demand for GenAI applications is power, space and cooling. To accommodate additional hardware in the existing setup, essentially increasing the IT density within a data center is a big task. Some may argue that the current state-of-the-art data center facilities are already optimized to such an extent that there is little to no room for further improvement. Then we’ll just have to go back to what started all of this- GenAI. Can we use GenAI to optimize the data center to a greater extent? The answer is most definitely a Yes.

In conclusion, the relationship between GenAI and its supporting underlying infrastructure is quite ambivalent. Certainly, Generative AI holds immense potential due to its wide range of applications. But, at the same time, we must use it responsibly given the type of impact it has on the underlying infrastructure and the environment. The best way to tackle this is by using GenAI or AI itself to optimize the infrastructure and to make the best use of our resources. Enterprises need to gear up to understand all the possible aspects of their GenAI adoption plan and strategy and give it a more holistic approach. The approaches should be flexible and must test the waters as this is a new technology with constant ongoing changes, to obtain the best possible outcome.

To learn more, please write to us at HCBU-PMG@hcltech.com.

Ref:

• https://www.mckinsey.com/industries/technology-media-and-telecommunications/our-insights/beyond-the-hype-capturing-the-potential-of-ai-and-gen-ai-in-tmt
• https://www.prnewswire.com/news-releases/generative-ai-market-to-be-worth-109-37-billion-by-2030-grand-view-research-inc-301718827.html#:~:text=SAN%20FRANCISCO%2C%20Jan.,34.6%25%20from%202022%20to%202030
• https://www.statista.com/outlook/tmo/artificial-intelligence/generative-ai/worldwide
• https://www.turing.com/resources/generative-ai-enhances-cybersecurity
• https://tmotions.com/applications-of-generative-ai-in-it-infrastructure-optimization/
• https://www.datacenterknowledge.com/industry-perspectives/rise-generative-ai-and-its-impact-data-center-sector
• https://pisrt.org/psr-press/journals/easl-vol-3-issue-2-2020/new-perspectives-on-internet-electricity-use-in-2030/
• https://www.forbes.com/sites/tiriasresearch/2023/05/12/generative-ai-breaks-the-data-center-data-center-infrastructure-and-operating-costs-projected-to-increase-to-over-76-billion-by-2028/?sh=522fb7d17c15
• https://medium.com/@kyeg/cloud-vs-on-premise-where-to-deploy-your-ai-applications-b584335ae86a
• https://medium.com/venture-beyond/on-ai-the-gpu-shortage-and-the-potential-of-decentralised-inference-and-training-networks-2442527af18e
• https://www.techtarget.com/searchnetworking/tip/The-role-of-generative-AI-in-networking#:~:text=How%20GenAI%20can%20support%20networking,workloads%2C%20even%20as%20staffing%20declines.