Raj Kapadia

Hi. Uh, this is Raj Kapadia, and I'm team lead in AI and ML section at Lutz and Kendall Ahmedabad. And I lead a team of 5 developer, and we build, you know, products around large language model and everything related to AI. And we have, you know, developed couple of SaaS products, uh, in this company as well. And, basically, I am electrical engineer, and I have master degree in electrical power system as well. But since, you know, my interest was in programming since I did my masters, Uh, I switched my career in while doing a job. I learned programming and did freelancing work in the space of, you know, major 3 areas. 1st is API development using Python and Flask and Node. Js and Express, both JavaScript and TypeScript. Then I have built, you know, applications around AI and ML, like, uh, you know, image classification or object detection, object segmentation, uh, in the space of natural language processing, like, um, text classification, uh, summarization, uh, question and answer on a document, so things like that. And, uh, the third one is chatbot development on Google Dialogflow. There are 2 platform, essential version and CX version. So I have experience with both, uh, platforms, and I have built around more than, I guess, 125 JetBots on both the platforms. And I have recently, you know, acquired I have recently acquired knowledge on the in the field of generative AI. So I have experience building, uh, uh, retrieval augmented generation or, you know, image generation or using using OpenAI assistant API or tool calling functionality, and I can, you know, build, uh, application around this, uh, LLM space as well. And recently, we are building, uh, kind of, uh, perplexity uh, SaaS platform in our current organization as well here in this part. We are using, you know, agents and tool calling functionality and, you know, indexing large amount of data and stuff like that. So, yeah, I guess, uh, this is this is my brief background.

Can you explain the concept of decorators in how you will you use them to implement logging mechanism? Uh, so, you know, uh, decorators are kind of, uh, you can in a in a plain term, it is a kind of a middleware. And, uh, when you provide a decorator on top of a function, that function will, you know, first run the decorator and then so Python, basically, will first run the decorator and then the actual function. And I haven't used, you know, uh, decorators for logging mechanism, what I in, uh, all my previous work. What we usually do is that, uh, let's say all my code is in source folder. So in that source folder, I create, uh, underscore, underscore, unit, underscore, underscore, pie file. And I write all my logging, uh, functionality into that in it file. Uh, usually, both, you know, standard output, uh, terminal output logging, and, uh, logging into log file. And then we can means I import all the, um, import logging or my logger into all the files in top in top, uh, when I need it. And then I use, you know, logging info, error, whichever I want to, and, um, do the logging. I guess this is what, uh, I have experience doing with.

Okay. How will you deploy and manage machine learning? Okay. So, or, you know, machine learning models, uh, are usually big in size. So if the model is big in size, then if we want to, you know, uh, create a container using Docker, then it takes time. So usually, what we do is that we keep the model in a separate space, like, let's say, uh, shared storage, and then, you know, provide the path of that model into our goal to environment variable. And, uh, so I guess AWS SageMaker and Google, uh, Cloud AI platform both provides, you know, hosted models as well. You build the model on SageMaker or on, uh, Google Cloud Machine Learning Platform. Form. You create your model, host the model, even both provides a facility to download the model as well. But if, you know, uh, if it is feasible to use the API version, then, you know, that's a better solution because, you know, we don't have to manage anything on our own. But if, you know, uh, we don't want to go on that route, what we can do is that we can download the model, put it on a cell, uh, storage, and get a part of that into our application through environment file, build the docker image, and, you know, run the docker image. And we can, you know, do the load load setting, load balancing, and everything that the ops can do so that, you know, our instance can be up in fraction of seconds. So I guess we can, uh, do this.

You explain also caching and how it is used in Python? So, uh, I guess, you know, we can use Radish to cache our information. Uh, all the routes in all the routes, we can, you know, cache the body and the response for that body. And we can look if the information is there, still there. If it is there, you know, we can send it back directly. Otherwise, you know, we can, uh, create that information. And, uh, you know, uh, we can use any queue based mechanism like, uh, bull m q or rabbit m q to, you know, maintain a queue. And, uh, if we talk about so there is another kind of caching as well. Uh, Python has, uh, I'm not sure, l caching or something. I I I actually don't remember the name, but there is a caching mechanism as well. So you can, you know, put a decorator on top of your function, and, uh, Python will, you know, cache the function input and output. Basically, hash it and store it in, um, cache so that, you know for example, let's say, you use, uh, search API. Let's say you use server API to search, uh, term. Let's say, uh, Apple iPhone 15 Max Pro, and, uh, you get bunch of results. And if you have used the caching decorator, what happens is that when you search the same term again, you know, you will get response in, like, 20 milliseconds. So I guess we can use these 2 caching mechanism.

Oh, can you explain the concept of async IO and how it differs from traditional setting in Python? I'm not 100% sure on this because I haven't basically used async IO in any of my application till now. But asynchronous code, basically, you know, certain uh, for example, certain function takes time to complete. And for example, let's say, uh, API route where you want to upload a document. And when you upload the document, obviously, it takes time to upload on, you know, s 3 or, uh, cloud storage, GCP cloud storage. And, uh, then you want to process it. Let's say you want to create vectors of the information inside the document, then it takes time. You, you know, chunk the information, get the embeddings from OpenAI embeddings. Let's just store the vectors into, uh, quadrant and, you know, reply back. So this whole process can take time. It also depends on the size of the document and the information inside the document as well. So, uh, for this kind of thing, uh, you you cannot, you know, let the front end wait the whole time. So what you can do is that you can, you know, immediately send the status back that, hey. We received the request, and this is the ID of the, uh, document that you have uploaded. And you can, you know, retrieve the ID. And using threading, you can, you know, create a new thread, do all the stuff I mentioned, and, you know, change the status of the document at every stage. So this is, uh, this this scenario explains threading, and asynchronous IO is a kind of asynchronous code. Uh, for example, let's say, uh, let's say you are using you are building Telegram bot and, uh, Telegram bot and in inside Telegram bot, you have some process that takes, let's say, 1 to 2 seconds or, let's say, not 1 to 2 second, but 1 to 2 minute to complete the process. So what you do is that, basically, you use async IO to, you know, let the process complete. Once the process is complete, you, you know, send a response back to the user. So, uh, I guess, async IO stays in the same function while threading, you know, takes the, uh, the process to, uh, different, uh, thread or different basically, different thread that is going to be, you know, invoked by a function as well. So it basically takes the process into separate path, completes the process, do it own work, while async IO stays on the same thread.

Okay. Can you explain the concept of transaction and how they are used in database? So, um, basically, transactions, uh, is a kind of, uh, you know, a way to make sure that the database thing or database action is successful. Uh, for example, let's say you want to create an row, uh, an item or a row inside, um, a table, and you try to, uh, insert the row into the table. So if that, you know, that is, uh, that is a success, then you, you know, commit that. Otherwise, you roll back to the previous state of the database. So in this way, transactions, you know, helps us, uh, make sure that either we, uh, insert neuro or we don't do that. So, you know, with the help of transactions, um, for example, in SQL, you use, you know, session rollback or session commit based on the, uh, accent, like, you want to create? So I guess, uh, this way we can, you know, handle the, uh, correctness of our accent using transactions.

Have you used any MLOps tools such as TensorFlow and? Can you give example on how you use each? Uh, I'm not sure. I've actually I guess, I don't understand the question because MLOps is related to logging all the, uh, all the actions that is being performed in our application or in our script of, you know, basically, from data pipeline to model pipeline to evolution pipeline to serving pipeline. Like, you get the data, clean the data, you know, uh, make it available for the model, train the model, evaluate the model. You know, let's say, train couple of models, evaluate all the couple of model and models and choose the best one and put it into the production. And you can do this, uh, uh, for multiple times you create the pipeline. So I guess MLOps is related to that, but, um, TensorFlow, PyTorch, and scikit learn are the framework to train, build, and inference, uh, machine learning models. So I have experience with all the 3 frameworks, TensorFlow, PyTorch, and scikit learn. I have experience building, you know, uh, CNN or image classification or object detection models using, uh, TensorFlow and PyTorch and all the you know, I I understand the whole framework of, uh, scikit learn and how it works, uh, uh, from, you know, uh, normalizing the data to, you know, encoding the data, categorical data to, you know, uh, filling the data, uh, splitting the data, training a model, let's say, random forest classifier log logistic regression or any model. And then, you know, uh, getting the scores and all the stuff I have, you know, experience with all the 3, uh, frameworks.

What best practices do you follow when writing unit and integration test for a Python machine learning pipeline? Uh, you know, the first thing I I should look into is I make sure that I have the correct model, uh, part by loading the model so that, you know, I don't face issue afterwards. Then what we can what I I do is that, basically, all the possible, uh, cases of the the model we are building. I I, you know, write all the test cases for that model. And, you know, uh, we can integrate into the deployment process as well. So until all the, uh, test cases are, you know, successfully checked, We don't move to the, uh, uh, further step, I guess. So these few things I do. Uh, I guess, also, we should look into the, uh, availability of the objects. For, for example, scaling, uh, the input that is coming via the request as well. So, you know, we we make sure that all the things are there before we do the predictions.

Demon said how you have used Docker and Kubernetes to improve the development workflow Python application. I guess I haven't used Kubernetes, but I I have used Docker, and, uh, I have been using, actually, Docker to deploy all the projects on Python or on Node. Js express as well. So, basically, what, uh, we do with Docker is that we write a Docker file. And if we need multiple images, we write Docker compose file. But often case is, you know, you your work is done with, uh, Docker file. You write a Docker file, you know, pull the Python image, uh, you know, upgrade pip, uh, copy the code, upgrade pip, uh, install the dependencies, and start the application using, you know, in case of Flask or in case of, uh, FastAPI. And then you, uh, run the docker image using Docker run command, and you can use hyphen d to, uh, use it in a detached mode. And you can, you know, see the logs using Docker logs hyphen f to follow the trail as well and provide the container ID. And I guess, uh, with these things, uh, you can, you know, easily manage, uh, easily manage the production of, uh, particular application. And, obviously, you push the code to GitHub and, you know, pull the latest code, build the image, and run the application again. Uh, also, you know, this whole process can be automated using GitLab as well. So in our company, there is a dev op guy who does the GitLab thing. So we, uh, push the code, uh, merge the code into the main branch. And when we create a new job, that job is created, first goes to the sandbox, then goes to the testing phase. If everything works well, then it, you know, replaces the, uh, production environment. So I guess we can use, you know, but I I actually haven't used Kubernetes ever. I guess, uh, that's it for this question. Thank you.