Genuine Exam Dumps For Databricks-Generative-AI-Engineer-Associate:
Prepare Yourself Expertly for Databricks-Generative-AI-Engineer-Associate Exam:
Our team of highly skilled and experienced professionals is dedicated to delivering up-to-date and precise study materials in PDF format to our customers. We deeply value both your time and financial investment, and we have spared no effort to provide you with the highest quality work. We ensure that our students consistently achieve a score of more than 95% in the Databricks Databricks-Generative-AI-Engineer-Associate exam. You provide only authentic and reliable study material. Our team of professionals is always working very keenly to keep the material updated. Hence, they communicate to the students quickly if there is any change in the Databricks-Generative-AI-Engineer-Associate dumps file. The Databricks Databricks-Generative-AI-Engineer-Associate exam question answers and Databricks-Generative-AI-Engineer-Associate dumps we offer are as genuine as studying the actual exam content.
24/7 Friendly Approach:
You can reach out to our agents at any time for guidance; we are available 24/7. Our agent will provide you information you need; you can ask them any questions you have. We are here to provide you with a complete study material file you need to pass your Databricks-Generative-AI-Engineer-Associate exam with extraordinary marks.
Quality Exam Dumps for Databricks Databricks-Generative-AI-Engineer-Associate:
Pass4surexams provide trusted study material. If you want to meet a sweeping success in your exam you must sign up for the complete preparation at Pass4surexams and we will provide you with such genuine material that will help you succeed with distinction. Our experts work tirelessly for our customers, ensuring a seamless journey to passing the Databricks Databricks-Generative-AI-Engineer-Associate exam on the first attempt. We have already helped a lot of students to ace IT certification exams with our genuine Databricks-Generative-AI-Engineer-Associate Exam Question Answers. Don't wait and join us today to collect your favorite certification exam study material and get your dream job quickly.
90 Days Free Updates for Databricks Databricks-Generative-AI-Engineer-Associate Exam Question Answers and Dumps:
Enroll with confidence at Pass4surexams, and not only will you access our comprehensive Databricks Databricks-Generative-AI-Engineer-Associate exam question answers and dumps, but you will also benefit from a remarkable offer – 90 days of free updates. In the dynamic landscape of certification exams, our commitment to your success doesn't waver. If there are any changes or updates to the Databricks Databricks-Generative-AI-Engineer-Associate exam content during the 90-day period, rest assured that our team will promptly notify you and provide the latest study materials, ensuring you are thoroughly prepared for success in your exam."
Databricks Databricks-Generative-AI-Engineer-Associate Real Exam Questions:
Quality is the heart of our service that's why we offer our students real exam questions with 100% passing assurance in the first attempt. Our Databricks-Generative-AI-Engineer-Associate dumps PDF have been carved by the experienced experts exactly on the model of real exam question answers in which you are going to appear to get your certification.
A Generative Al Engineer is tasked with developing an application that is based on an open sourcelarge language model (LLM). They need a foundation LLM with a large context window.Which model fits this need
A. DistilBERT B. MPT-30B C. Llama2-70B D. DBRX
Answer: C Explanation:Problem Context: The engineer needs an open-source LLM with a large context window to develop an application.Explanation of Options:Option A: DistilBERT: While an efficient and smaller version of BERT, DistilBERT does not provide a particularly large context window.Option B: MPT-30B: This model, while large, is not specified as being particularly notable for itscontext window capabilities.Option C: Llama2-70B: Known for its large model size and extensive capabilities, including a largecontext window. It is also available as an open-source model, making it ideal for applicationsrequiring extensive contextual understanding.Option D: DBRX: This is not a recognized standard model in the context of large language models with extensive context windows.Thus, Option C (Llama2-70B) is the best fit as it meets the criteria of having a large context windowand being available for open-source use, suitable for developing robust language understanding applications
Question # 2
A Generative AI Engineer received the following business requirements for an external chatbot.The chatbot needs to know what types of questions the user asks and routes to appropriate modelsto answer the questions. For example, the user might ask about upcoming event details. Anotheruser might ask about purchasing tickets for a particular event.What is an ideal workflow for such a chatbot?
A. The chatbot should only look at previous event information B. There should be two different chatbots handling different types of user queries. C. The chatbot should be implemented as a multi-step LLM workflow. First, identify the type ofquestion asked, then route the question to the appropriate model. If its an upcoming eventquestion, send the query to a text-to-SQL model. If its about ticket purchasing, the customer shouldbe redirected to a payment platform. D. The chatbot should only process payments
Answer: C Explanation:Problem Context: The chatbot must handle various types of queries and intelligently route them tothe appropriate responses or systems.Explanation of Options:Option A: Limiting the chatbot to only previous event information restricts its utility and does notmeet the broader business requirements.Option B: Having two separate chatbots could unnecessarily complicate user interaction and increasemaintenance overhead.Option C: Implementing a multi-step workflow where the chatbot first identifies the type of questionand then routes it accordingly is the most efficient and scalable solution. This approach allows thechatbot to handle a variety of queries dynamically, improving user experience and operational efficiency.Option D: Focusing solely on payments would not satisfy all the specified user interaction needs,such as inquiring about event details.Option C offers a comprehensive workflow that maximizes the chatbots utility and responsiveness todifferent user needs, aligning perfectly with the business requirements.
Question # 3
A Generative AI Engineer is building a RAG application that will rely on context retrieved from sourcedocuments that are currently in PDF format. These PDFs can contain both text and images. They wantto develop a solution using the least amount of lines of code.Which Python package should be used to extract the text from the source documents?
A. flask B. beautifulsoup C. unstructured D. numpy
Answer: C Explanation:Problem Context: The engineer needs to extract text from PDF documents, which may contain bothtext and images. The goal is to find a Python package that simplifies this task using the least amountof code.Explanation of Options:Option A: flask: Flask is a web framework for Python, not suitable for processing or extracting contentfrom PDFs.Option B: beautifulsoup: Beautiful Soup is designed for parsing HTML and XML documents, not PDFs.Option C: unstructured: This Python package is specifically designed to work with unstructured data,including extracting text from PDFs. It provides functionalities to handle various types of content indocuments with minimal coding, making it ideal for the task.Option D: numpy: Numpy is a powerful library for numerical computing in Python and does notprovide any tools for text extraction from PDFs. Given the requirement, Option C (unstructured) is the most appropriate as it directly addresses theneed to efficiently extract text from PDF documents with minimal code.
Question # 4
A team wants to serve a code generation model as an assistant for their software developers. Itshould support multiple programming languages. Quality is the primary objective.Which of the Databricks Foundation Model APIs, or models available in the Marketplace, would b the best fit?
A. Llama2-70b B. BGE-large C. MPT-7b D. CodeLlama-34B
Answer: D Explanation:For a code generation model that supports multiple programming languages and where quality is theprimary objective, CodeLlama-34B is the most suitable choice. Heres the reasoning:Specialization in Code Generation:CodeLlama-34B is specifically designed for code generation tasks. This model has been trained with afocus on understanding and generating code, which makes it particularly adept at handling variousprogramming languages and coding contexts.Capacity and Performance:The "34B" indicates a model size of 34 billion parameters, suggesting a high capacity for handlingcomplex tasks and generating high-quality outputs. The large model size typically correlates withbetter understanding and generation capabilities in diverse scenarios.Suitability for Development Teams:Given that the model is optimized for code, it will be able to assist software developers moreeffectively than general-purpose models. It understands coding syntax, semantics, and the nuancesof different programming languages.Why Other Options Are Less Suitable:A (Llama2-70b): While also a large model, it's more general-purpose and may not be as fine-tunedfor code generation as CodeLlama.B (BGE-large): This model may not specifically focus on code generation.C (MPT-7b): Smaller than CodeLlama-34B and likely less capable in handling complex codegeneration tasks at high quality.Therefore, for a high-quality, multi-language code generation application, CodeLlama-34B (option D)is the best fit.
Question # 5
A Generative AI Engineer is designing a chatbot for a gaming company that aims to engage users onits platform while its users play online video games.Which metric would help them increase user engagement and retention for their platform?
A. Randomness B. Diversity of responses C. Lack of relevance D. Repetition of responses
Answer: B Explanation:In the context of designing a chatbot to engage users on a gaming platform, diversity of responses(option B) is a key metric to increase user engagement and retention. Heres why:Diverse and Engaging Interactions:A chatbot that provides varied and interesting responses will keep users engaged, especially in aninteractive environment like a gaming platform. Gamers typically enjoy dynamic and evolvingconversations, and diversity of responses helps prevent monotony, encouraging users to interactmore frequently with the bot.Increasing Retention:By offering different types of responses to similar queries, the chatbot can create a sense of noveltyand excitement, which enhances the users experience and makes them more likely to return to theplatform.Why Other Options Are Less Effective:A (Randomness): Random responses can be confusing or irrelevant, leading to frustration andreducing engagement.C (Lack of Relevance): If responses are not relevant to the users queries, this will degrade the userexperience and lead to disengagement.D (Repetition of Responses): Repetitive responses can quickly bore users, making the chatbot feeluninteresting and reducing the likelihood of continued interaction.Thus, diversity of responses (option B) is the most effective way to keep users engaged and retainthem on the platform.
Question # 6
A Generative AI Engineer is creating an LLM-powered application that will need access to up-to-datenews articles and stock prices.The design requires the use of stock prices which are stored in Delta tables and finding the latestrelevant news articles by searching the internet.How should the Generative AI Engineer architect their LLM system?
A. Use an LLM to summarize the latest news articles and lookup stock tickers from the summaries to find stock prices. B. Query the Delta table for volatile stock prices and use an LLM to generate a search query toinvestigate potential causes of the stock volatility. C. Download and store news articles and stock price information in a vector store. Use a RAGarchitecture to retrieve and generate at runtime. D. Create an agent with tools for SQL querying of Delta tables and web searching, provide retrievedvalues to an LLM for generation of response.
Answer: D Explanation:To build an LLM-powered system that accesses up-to-date news articles and stock prices, the bestapproach is to create an agent that has access to specific tools (option D).Agent with SQL and Web Search Capabilities:By using an agent-based architecture, the LLM can interact with external tools. The agent can queryDelta tables (for up-to-date stock prices) via SQL and perform web searches to retrieve the latestnews articles. This modular approach ensures the system can access both structured (stock prices)and unstructured (news) data sources dynamically.Why This Approach Works:SQL Queries for Stock Prices: Delta tables store stock prices, which the agent can query directly forthe latest data.Web Search for News: For news articles, the agent can generate search queries and retrieve the mostrelevant and recent articles, then pass them to the LLM for processing.Why Other Options Are Less Suitable:A (Summarizing News for Stock Prices): This convoluted approach would not ensure accuracy whenretrieving stock prices, which are already structured and stored in Delta tables.B (Stock Price Volatility Queries): While this could retrieve relevant information, it doesn't addresshow to obtain the most up-to-date news articles.C (Vector Store): Storing news articles and stock prices in a vector store might not capture the realtimenature of stock data and news updates, as it relies on pre-existing data rather than dynamicquerying.Thus, using an agent with access to both SQL for querying stock prices and web search for retrievingnews articles is the best approach for ensuring up-to-date and accurate responses.
Question # 7
A Generative AI Engineer is building an LLM to generate article summaries in the form of a type ofpoem, such as a haiku, given the article content. However, the initial output from the LLM does notmatch the desired tone or style.Which approach will NOT improve the LLMs response to achieve the desired response?
A. Provide the LLM with a prompt that explicitly instructs it to generate text in the desired tone and style B. Use a neutralizer to normalize the tone and style of the underlying documents C. Include few-shot examples in the prompt to the LLM D. Fine-tune the LLM on a dataset of desired tone and style
Answer: B Explanation:The task at hand is to improve the LLMs ability to generate poem-like article summaries with thedesired tone and style. Using a neutralizer to normalize the tone and style of the underlyingdocuments (option B) will not help improve the LLMs ability to generate the desired poetic style.Heres why:Neutralizing Underlying Documents:A neutralizer aims to reduce or standardize the tone of input data. However, this contradicts the goal,which is to generate text with a specific tone and style (like haikus). Neutralizing the sourcedocuments will strip away the richness of the content, making it harder for the LLM to generatecreative, stylistic outputs like poems.Why Other Options Improve Results:A (Explicit Instructions in the Prompt): Directly instructing the LLM to generate text in a specific toneand style helps align the output with the desired format (e.g., haikus). This is a common and effectivetechnique in prompt engineering.C (Few-shot Examples): Providing examples of the desired output format helps the LLM understandthe expected tone and structure, making it easier to generate similar outputs.D (Fine-tuning the LLM): Fine-tuning the model on a dataset that contains examples of the desiredtone and style is a powerful way to improve the models ability to generate outputs that match thetarget format.Therefore, using a neutralizer (option B) is not an effective method for achieving the goal ofgenerating stylized poetic summaries.
Question # 8
A Generative AI Engineer developed an LLM application using the provisioned throughputFoundation Model API. Now that the application is ready to be deployed, they realize their volume ofrequests are not sufficiently high enough to create their own provisioned throughput endpoint. Theywant to choose a strategy that ensures the best cost-effectiveness for their application.What strategy should the Generative AI Engineer use?
A. Switch to using External Models instead B. Deploy the model using pay-per-token throughput as it comes with cost guarantees C. Change to a model with a fewer number of parameters in order to reduce hardware constraintissues D. Throttle the incoming batch of requests manually to avoid rate limiting issues
Answer: B Explanation:Problem Context: The engineer needs a cost-effective deployment strategy for an LLM applicationwith relatively low request volume.Explanation of Options:Option A: Switching to external models may not provide the required control or integrationnecessary for specific application needs.Option B: Using a pay-per-token model is cost-effective, especially for applications with variable orlow request volumes, as it aligns costs directly with usage.Option C: Changing to a model with fewer parameters could reduce costs, but might also impact theperformance and capabilities of the application.Option D: Manually throttling requests is a less efficient and potentially error-prone strategy formanaging costs.Option B is ideal, offering flexibility and cost control, aligning expenses directly with the application'susage patterns.
Question # 9
A Generative Al Engineer has already trained an LLM on Databricks and it is now ready to bedeployed.Which of the following steps correctly outlines the easiest process for deploying a model onDatabricks?
A. Log the model as a pickle object, upload the object to Unity Catalog Volume, register it to UnityCatalog using MLflow, and start a serving endpoint B. Log the model using MLflow during training, directly register the model to Unity Catalog using theMLflow API, and start a serving endpoint C. Save the model along with its dependencies in a local directory, build the Docker image, and runthe Docker container D. Wrap the LLMs prediction function into a Flask application and serve using Gunicorn
Answer: B Explanation:Problem Context: The goal is to deploy a trained LLM on Databricks in the simplest and mostintegrated manner.Explanation of Options:Option A: This method involves unnecessary steps like logging the model as a pickle object, which isnot the most efficient path in a Databricks environment.Option B: Logging the model with MLflow during training and then using MLflows API to register andstart serving the model is straightforward and leverages Databricks built-in functionalities forseamless model deployment.Option C: Building and running a Docker container is a complex and less integrated approach withinthe Databricks ecosystem.Option D: Using Flask and Gunicorn is a more manual approach and less integrated compared to thenative capabilities of Databricks and MLflow.Option B provides the most straightforward and efficient process, utilizing Databricks ecosystem toits full advantage for deploying models
Question # 10
A Generative AI Engineer has created a RAG application which can help employees retrieve answersfrom an internal knowledge base, such as Confluence pages or Google Drive. The prototypeapplication is now working with some positive feedback from internal company testers. Now theGenerative Al Engineer wants to formally evaluate the systems performance and understand whereto focus their efforts to further improve the system.How should the Generative AI Engineer evaluate the system
A. Use cosine similarity score to comprehensively evaluate the quality of the final generatedanswers. B. Curate a dataset that can test the retrieval and generation components of the system separately.Use MLflows built in evaluation metrics to perform the evaluation on the retrieval and generationcomponents. C. Benchmark multiple LLMs with the same data and pick the best LLM for the job. D. Use an LLM-as-a-judge to evaluate the quality of the final answers generated.
Answer: B Explanation:Problem Context: After receiving positive feedback for the RAG application prototype, the next stepis to formally evaluate the system to pinpoint areas for improvement.Explanation of Options:Option A: While cosine similarity scores are useful, they primarily measure similarity rather than theoverall performance of an RAG system.Option B: This option provides a systematic approach to evaluation by testing both retrieval andgeneration components separately. This allows for targeted improvements and a clear understandingof each component's performance, using MLflows metrics for a structured and standardizedassessment.Option C: Benchmarking multiple LLMs does not focus on evaluating the existing systemscomponents but rather on comparing different models.Option D: Using an LLM as a judge is subjective and less reliable for systematic performance evaluation.Option B is the most comprehensive and structured approach, facilitating precise evaluations andimprovements on specific components of the RAG system
Question # 11
A Generative Al Engineer has successfully ingested unstructured documents and chunked them bydocument sections. They would like to store the chunks in a Vector Search index. The current formatof the dataframe has two columns: (i) original document file name (ii) an array of text chunks foreach document.What is the most performant way to store this dataframe?
A. Split the data into train and test set, create a unique identifier for each document, then save to aDelta table B. Flatten the dataframe to one chunk per row, create a unique identifier for each row, and save to aDelta table C. First create a unique identifier for each document, then save to a Delta table D. Store each chunk as an independent JSON file in Unity Catalog Volume. For each JSON file, the keyis the document section name and the value is the array of text chunks for that section
Answer: B Explanation:Problem Context: The engineer needs an efficient way to store chunks of unstructured documents tofacilitate easy retrieval and search. The current dataframe consists of document filenames andassociated text chunks.Explanation of Options:Option A: Splitting into train and test sets is more relevant for model training scenarios and notdirectly applicable to storage for retrieval in a Vector Search index.Option B: Flattening the dataframe such that each row contains a single chunk with a uniqueidentifier is the most performant for storage and retrieval. This structure aligns well with how data isindexed and queried in vector search applications, making it easier to retrieve specific chunksefficiently.Option C: Creating a unique identifier for each document only does not address the need to access
Question # 12
A Generative Al Engineer is building a RAG application that answers questions about internaldocuments for the company SnoPen AI.The source documents may contain a significant amount of irrelevant content, such asadvertisements, sports news, or entertainment news, or content about other companies.Which approach is advisable when building a RAG application to achieve this goal of filteringirrelevant information
A. Keep all articles because the RAG application needs to understand non-company content to avoidanswering questions about them. B. Include in the system prompt that any information it sees will be about SnoPenAI, even if no datafiltering is performed. C. Include in the system prompt that the application is not supposed to answer any questionsunrelated to SnoPen Al. D. Consolidate all SnoPen AI related documents into a single chunk in the vector database.
Answer: C Explanation:In a Retrieval-Augmented Generation (RAG) application built to answer questions about internaldocuments, especially when the dataset contains irrelevant content, it's crucial to guide the systemto focus on the right information. The best way to achieve this is by including a clear instruction inthe system prompt (option C).System Prompt as Guidance:The system prompt is an effective way to instruct the LLM to limit its focus to SnoPen AI-related
content. By clearly specifying that the model should avoid answering questions unrelated to SnoPenAI, you add an additional layer of control that helps the model stay on-topic, even if irrelevantcontent is present in the dataset.Why This Approach Works:The prompt acts as a guiding principle for the model, narrowing its focus to specific domains. Thisprevents the model from generating answers based on irrelevant content, such as advertisements ornews unrelated to SnoPen AI.Why Other Options Are Less Suitable:A (Keep All Articles): Retaining all content, including irrelevant materials, without any filtering makes the system prone to generating answers based on unwanted data.B (Include in the System Prompt about SnoPen AI): This option doesnt address irrelevant contentdirectly, and without filtering, the model might still retrieve and use irrelevant data.D (Consolidating Documents into a Single Chunk): Grouping documents into a single chunk makesthe retrieval process less efficient and wont help filter out irrelevant content effectively.Therefore, instructing the system in the prompt not to answer questions unrelated to SnoPen AI(option C) is the best approach to ensure the system filters out irrelevant information.the system prone to generating answers based on unwanted data.B (Include in the System Prompt about SnoPen AI): This option doesnt address irrelevant contentdirectly, and without filtering, the model might still retrieve and use irrelevant data.D (Consolidating Documents into a Single Chunk): Grouping documents into a single chunk makesthe retrieval process less efficient and wont help filter out irrelevant content effectively.Therefore, instructing the system in the prompt not to answer questions unrelated to SnoPen AI(option C) is the best approach to ensure the system filters out irrelevant information.
Question # 13
A Generative Al Engineer is building a system which will answer questions on latest stock newsarticles.Which will NOT help with ensuring the outputs are relevant to financial news
A. Implement a comprehensive guardrail framework that includes policies for content filters tailoredto the finance sector B. Increase the compute to improve processing speed of questions to allow greater relevancyanalysis C Implement a profanity filter to screen out offensive language D. Incorporate manual reviews to correct any problematic outputs prior to sending to the users
Answer: B Explanation:In the context of ensuring that outputs are relevant to financial news, increasing compute power(option B) does not directly improve the relevance of the LLM-generated outputs. Heres why:Compute Power and Relevancy:Increasing compute power can help the model process inputs faster, but it does not inherentlyimprove the relevance of the answers. Relevancy depends on the data sources, the retrieval method,and the filtering mechanisms in place, not on how quickly the model processes the query.What Actually Helps with Relevance:Other methods, like content filtering, guardrails, or manual review, can directly impact the relevanceof the models responses by ensuring the model focuses on pertinent financial content. Thesemethods help tailor the LLMs responses to the financial domain and avoid irrelevant or harmfuloutputs.Why Other Options Are More Relevant:A (Comprehensive Guardrail Framework): This will ensure that the model avoids generating contentthat is irrelevant or inappropriate in the finance sector.C (Profanity Filter): While not directly related to financial relevancy, ensuring the output is clean andprofessional is still important in maintaining the quality of responses.D (Manual Review): Incorporating human oversight to catch and correct issues with the LLMs outputensures the final answers are aligned with financial content expectations.
Question # 14
A Generative AI Engineer is developing a patient-facing healthcare-focused chatbot. If the patientsquestion is not a medical emergency, the chatbot should solicit more information from the patient topass to the doctors office and suggest a few relevant pre-approved medical articles for reading. Ifthe patients question is urgent, direct the patient to calling their local emergency services.Given the following user input:œI have been experiencing severe headaches and dizziness for the past two days.Which response is most appropriate for the chatbot to generate?
A. Here are a few relevant articles for your browsing. Let me know if you have questions after readingthem. B. Please call your local emergency services. C. Headaches can be tough. Hope you feel better soon! D. Please provide your age, recent activities, and any other symptoms you have noticed along withyour headaches and dizziness.
Answer: B Explanation:Problem Context: The task is to design responses for a healthcare-focused chatbot that appropriatelyaddresses the urgency of a patient's symptoms.Explanation of Options:Option A: Suggesting articles might be suitable for less urgent inquiries but is inappropriate forsymptoms that could indicate a serious condition.Option B: Given the description of severe symptoms like headaches and dizziness, directing thepatient to emergency services is prudent. This aligns with medical guidelines that recommendimmediate professional attention for such severe symptoms.Option C: Offering well-wishes does not address the potential seriousness of the symptoms and lacksappropriate action.Option D: While gathering more information is part of a detailed assessment, the immediate needhere suggests a more urgent response.Given the potential severity of the described symptoms, Option B is the most appropriate, ensuringthe chatbot directs patients to seek urgent care when needed, potentially saving lives
Question # 15
Which indicator should be considered to evaluate the safety of the LLM outputs when qualitativelyassessing LLM responses for a translation use case?
A. The ability to generate responses in code B. The similarity to the previous language C. The latency of the response and the length D. The accuracy and relevance of the responses
Answer: D Explanation:Problem Context: When assessing the safety and effectiveness of LLM outputs in a translation usecase, it is essential to ensure that the translations accurately and relevantly convey the intendedmessage. The evaluation should focus on how well the LLM understands and processes differentlanguages and contexts.Explanation of Options:Option A: The ability to generate responses in code “ This is not relevant to translation quality orsafety.Option B: The similarity to the previous language “ While ensuring that translations preserve theoriginal's intent is important, this doesn't directly address the overall quality or safety of thetranslation.Option C: The latency of the response and the length of text generated “ These operational metricsare less critical in assessing the qualitative aspects of translation safety.Option D: The accuracy and relevance of the responses “ This is crucial in translation to ensure thatthe translated content is true to the original in meaning and appropriateness. Accuracy andrelevance directly impact the effectiveness and safety of translations, especially in sensitive ornuanced contexts
Question # 16
A Generative AI Engineer is developing an LLM application that users can use to generatepersonalized birthday poems based on their names.Which technique would be most effective in safeguarding the application, given the potential formalicious user inputs?
A. Implement a safety filter that detects any harmful inputs and ask the LLM to respond that it isunable to assist B. Reduce the time that the users can interact with the LLM C. Ask the LLM to remind the user that the input is malicious but continue the conversation with theuser D. Increase the amount of compute that powers the LLM to process input faster
Answer: A Explanation:In this case, the Generative AI Engineer is developing an application to generate personalizedbirthday poems, but theres a need to safeguard against malicious user inputs. The best solution is toimplement a safety filter (option A) to detect harmful or inappropriate inputs.Safety Filter Implementation:Safety filters are essential for screening user input and preventing inappropriate content from beingprocessed by the LLM. These filters can scan inputs for harmful language, offensive terms, ormalicious content and intervene before the prompt is passed to the LLM.Graceful Handling of Harmful Inputs:Once the safety filter detects harmful content, the system can provide a message to the user, such as"I'm unable to assist with this request," instead of processing or responding to malicious input. Thisprotects the system from generating harmful content and ensures a controlled interactionenvironment.Why Other Options Are Less Suitable B (Reduce Interaction Time): Reducing the interaction time wont prevent malicious inputs frombeing entered.C (Continue the Conversation): While its possible to acknowledge malicious input, it is not safe tocontinue the conversation with harmful content. This could lead to legal or reputational risks.D (Increase Compute Power): Adding more compute doesnt address the issue of harmful contentand would only speed up processing without resolving safety concerns.Therefore, implementing a safety filter that blocks harmful inputs is the most effective technique forsafeguarding the application.
Question # 17
What is an effective method to preprocess prompts using custom code before sending them to anLLM?
A. Directly modify the LLMs internal architecture to include preprocessing steps B. It is better not to introduce custom code to preprocess prompts as the LLM has not been trainedwith examples of the preprocessed prompts C. Rather than preprocessing prompts, its more effective to postprocess the LLM outputs to align theoutputs to desired outcomes D. Write a MLflow PyFunc model that has a separate function to process the prompts
Answer: D Explanation:The most effective way to preprocess prompts using custom code is to write a custom model, such asan MLflow PyFunc model. Heres a breakdown of why this is the correct approach:MLflow PyFunc Models:MLflow is a widely used platform for managing the machine learning lifecycle, includingexperimentation, reproducibility, and deployment. A PyFunc model is a generic Python functionmodel that can implement custom logic, which includes preprocessing prompts.Preprocessing Prompts:Preprocessing could include various tasks like cleaning up the user input, formatting it according tospecific rules, or augmenting it with additional context before passing it to the LLM. Writing thispreprocessing as part of a PyFunc model allows the custom code to be managed, tested, anddeployed easily.Modular and Reusable:By separating the preprocessing logic into a PyFunc model, the system becomes modular, making iteasier to maintain and update without needing to modify the core LLM or retrain it.Why Other Options Are Less Suitable:A (Modify LLMs Internal Architecture): Directly modifying the LLM's architecture is highly impracticaland can disrupt the models performance. LLMs are typically treated as black-box models for taskslike prompt processing B (Avoid Custom Code): While its true that LLMs haven't been explicitly trained with preprocessedprompts, preprocessing can still improve clarity and alignment with desired input formats withoutconfusing the model.C (Postprocessing Outputs): While postprocessing the output can be useful, it doesn't address theneed for clean and well-formatted inputs, which directly affect the quality of the model's responses.Thus, using an MLflow PyFunc model allows for flexible and controlled preprocessing of prompts in ascalable way, making it the most effective method.
Question # 18
A Generative AI Engineer wants to build an LLM-based solution to help a restaurant improve itsonline customer experience with bookings by automatically handling common customer inquiries.The goal of the solution is to minimize escalations to human intervention and phone calls whilemaintaining a personalized interaction. To design the solution, the Generative AI Engineer needs todefine the input data to the LLM and the task it should perform.Which input/output pair will support their goal?
A. Input: Online chat logs; Output: Group the chat logs by users, followed by summarizing each usersinteractions B. Input: Online chat logs; Output: Buttons that represent choices for booking details C. Input: Customer reviews; Output: Classify review sentiment D. Input: Online chat logs; Output: Cancellation options
Answer: B Explanation:ï‚? Context: The goal is to improve the online customer experience in a restaurant by handling commoninquiries about bookings, minimizing escalations, and maintaining personalized interactions.ï‚? Explanation of Options:Option A: Grouping and summarizing chat logs by user could provide insights into customerinteractions but does not directly address the task of handling booking inquiries or minimizingescalations.Option B: Using chat logs to generate interactive buttons for booking details directly supports thegoal of facilitating online bookings, minimizing the need for human intervention by providing clear,interactive options for customers to self-serve.Option C: Classifying sentiment of customer reviews does not directly help with booking inquiries,although it might provide valuable feedback insights.Option D: Providing cancellation options is helpful but narrowly focuses on one aspect of the bookingprocess and doesn't support the broader goal of handling common inquiries about bookings.Option B best supports the goal of improving online interactions by using chat logs to generateactionable items for customers, helping them complete booking tasks efficiently and reducing theneed for human intervention.
Question # 19
A Generative AI Engineer is tasked with deploying an application that takes advantage of a customMLflow Pyfunc model to return some interim results.How should they configure the endpoint to pass the secrets and credentials?
A. Use spark.conf.set () B. Pass variables using the Databricks Feature Store API C. Add credentials using environment variables D. Pass the secrets in plain text
Answer: C Explanation:ï‚? Context: Deploying an application that uses an MLflow Pyfunc model involves managing sensitiveinformation such as secrets and credentials securely.ï‚? Explanation of Options:Option A: Use spark.conf.set(): While this method can pass configurations within Spark jobs, using itfor secrets is not recommended because it may expose them in logs or Spark UI.Option B: Pass variables using the Databricks Feature Store API: The Feature Store API is designed formanaging features for machine learning, not for handling secrets or credentials.Option C: Add credentials using environment variables: This is a common practice for managingcredentials in a secure manner, as environment variables can be accessed securely by applicationswithout exposing them in the codebase.Option D: Pass the secrets in plain text: This is highly insecure and not recommended, as it exposessensitive information directly in the code.Therefore, Option C is the best method for securely passing secrets and credentials to an application,protecting them from exposure.
Question # 20
A Generative AI Engineer is developing a chatbot designed to assist users with insurance-relatedqueries. The chatbot is built on a large language model (LLM) and is conversational. However, tomaintain the chatbots focus and to comply with company policy, it must not provide responses toquestions about politics. Instead, when presented with political inquiries, the chatbot shouldrespond with a standard message:œSorry, I cannot answer that. I am a chatbot that can only answer questions around insurance.Which framework type should be implemented to solve this?
A. Safety Guardrail B. Security Guardrail C. Contextual Guardrail D. Compliance Guardrail
Answer: A Explanation:In this scenario, the chatbot must avoid answering political questions and instead provide a standardmessage for such inquiries. Implementing a Safety Guardrail is the appropriate solution for this:What is a Safety Guardrail?Safety guardrails are mechanisms implemented in Generative AI systems to ensure the modelbehaves within specific bounds. In this case, it ensures the chatbot does not answer politicallysensitive or irrelevant questions, which aligns with the business rules.Preventing Responses to Political Questions:The Safety Guardrail is programmed to detect specific types of inquiries (like political questions) andprevent the model from generating responses outside its intended domain. When such queries aredetected, the guardrail intervenes and provides a pre-defined response: œSorry, I cannot answer that.I am a chatbot that can only answer questions around insurance.How It Works in Practice:The LLM system can include a classification layer or trigger rules based on specific keywords relatedto politics. When such terms are detected, the Safety Guardrail blocks the normal generation flowand responds with the fixed message Why Other Options Are Less Suitable:B (Security Guardrail): This is more focused on protecting the system from security vulnerabilities ordata breaches, not controlling the conversational focus.C (Contextual Guardrail): While context guardrails can limit responses based on context, safetyguardrails are specifically about ensuring the chatbot stays within a safe conversational scope.D (Compliance Guardrail): Compliance guardrails are often related to legal and regulatory adherence,which is not directly relevant here.Therefore, a Safety Guardrail is the right framework to ensure the chatbot only answers insurancerelatedqueries and avoids political discussions.
Question # 21
Generative AI Engineer at an electronics company just deployed a RAG application for customers toask questions about products that the company carries. However, they received feedback that theRAG response often returns information about an irrelevant product.What can the engineer do to improve the relevance of the RAGs response?
A. Assess the quality of the retrieved context B. Implement caching for frequently asked questions C. Use a different LLM to improve the generated response D. Use a different semantic similarity search algorithm
Answer: A Explanation:In a Retrieval-Augmented Generation (RAG) system, the key to providing relevant responses lies inthe quality of the retrieved context. Heres why option A is the most appropriate solution:Context Relevance:The RAG model generates answers based on retrieved documents or context. If the retrievedinformation is about an irrelevant product, it suggests that the retrieval step is failing to select theright context. The Generative AI Engineer must first assess the quality of what is being retrieved andensure it is pertinent to the query.Vector Search and Embedding Similarity:RAG typically uses vector search for retrieval, where embeddings of the query are matched againstembeddings of product descriptions. Assessing the semantic similarity search process ensures thatthe closest matches are actually relevant to the query.Fine-tuning the Retrieval Process:By improving the retrieval quality, such as tuning the embeddings or adjusting the retrieval strategy,the system can return more accurate and relevant product information.Why Other Options Are Less Suitable:B (Caching FAQs): Caching can speed up responses for frequently asked questions but wont improvethe relevance of the retrieved content for less frequent or new queries.C (Use a Different LLM): Changing the LLM only affects the generation step, not the retrieval process,which is the core issue here. D (Different Semantic Search Algorithm): This could help, but the first step is to evaluate the currentretrieval context before replacing the search algorithm.Therefore, improving and assessing the quality of the retrieved context (option A) is the first step tofixing the issue of irrelevant product information.
Question # 22
A Generative AI Engineer has been asked to design an LLM-based application that accomplishes thefollowing business objective: answer employee HR questions using HR PDF documentation.Which set of high level tasks should the Generative AI Engineer's system perform?
A. Calculate averaged embeddings for each HR document, compare embeddings to user query tofind the best document. Pass the best document with the user query into an LLM with a large contextwindow to generate a response to the employee. B. Use an LLM to summarize HR documentation. Provide summaries of documentation and userquery into an LLM with a large context window to generate a response to the user. C. Create an interaction matrix of historical employee questions and HR documentation. Use ALS tofactorize the matrix and create embeddings. Calculate the embeddings of new queries and use themto find the best HR documentation. Use an LLM to generate a response to the employee questionbased upon the documentation retrieved. D. Split HR documentation into chunks and embed into a vector store. Use the employee question toretrieve best matched chunks of documentation, and use the LLM to generate a response to theemployee based upon the documentation retrieved.
Answer: D Explanation:To design an LLM-based application that can answer employee HR questions using HR PDFdocumentation, the most effective approach is option D. Here's why:Chunking and Vector Store Embedding:HR documentation tends to be lengthy, so splitting it into smaller, manageable chunks helps optimizeretrieval. These chunks are then embedded into a vector store (a database that stores vectorrepresentations of text). Each chunk of text is transformed into an embedding using a transformerbasedmodel, which allows for efficient similarity-based retrieval.Using Vector Search for Retrieval:When an employee asks a question, the system converts their query into an embedding as well. Thisembedding is then compared with the embeddings of the document chunks in the vector store. Themost semantically similar chunks are retrieved, which ensures that the answer is based on the mostrelevant parts of the documentation.LLM to Generate a Response:Once the relevant chunks are retrieved, these chunks are passed into the LLM, which uses them ascontext to generate a coherent and accurate response to the employee's question.Why Other Options Are Less Suitable:A (Calculate Averaged Embeddings): Averaging embeddings might dilute important information. Itdoesn't provide enough granularity to focus on specific sections of documents.B (Summarize HR Documentation): Summarization loses the detail necessary for HR-related queries,which are often specific. It would likely miss the mark for more detailed inquiries.C (Interaction Matrix and ALS): This approach is better suited for recommendation systems and not for HR queries, as its focused on collaborative filtering rather than text-based retrieval.Thus, option D is the most effective solution for providing precise and contextual answers based onHR documentation.
Question # 23
A Generative AI Engineer is creating an agent-based LLM system for their favorite monster truckteam. The system can answer text based questions about the monster truck team, lookup eventdates via an API call, or query tables on the teams latest standings.How could the Generative AI Engineer best design these capabilities into their system?
A. Ingest PDF documents about the monster truck team into a vector store and query it in a RAGarchitecture. B. Write a system prompt for the agent listing available tools and bundle it into an agent system thatruns a number of calls to solve a query C. Instruct the LLM to respond with œRAG , œAPI , or œTABLE depending on the query, then use textparsing and conditional statements to resolve the query. D. Build a system prompt with all possible event dates and table information in the system prompt.Use a RAG architecture to lookup generic text questions and otherwise leverage the information inthe system prompt.
Answer: B Explanation:In this scenario, the Generative AI Engineer needs to design a system that can handle different typesof queries about the monster truck team. The queries may involve text-based information, APIlookups for event dates, or table queries for standings. The best solution is to implement a toolbasedagent system.Heres how option B works, and why its the most appropriate answer:System Design Using Agent-Based Model:In modern agent-based LLM systems, you can design a system where the LLM (Large LanguageModel) acts as a central orchestrator. The model can "decide" which tools to use based on the query.These tools can include API calls, table lookups, or natural language searches. The system shouldcontain a system prompt that informs the LLM about the available tools.System Prompt Listing Tools:By creating a well-crafted system prompt, the LLM knows which tools are at its disposal. For instance,one tool may query an external API for event dates, another might look up standings in a database,and a third may involve searching a vector database for general text-based information. The agentwill be responsible for calling the appropriate tool depending on the query.Agent Orchestration of Calls:The agent system is designed to execute a series of steps based on the incoming query. If a user asksfor the next event date, the system will recognize this as a task that requires an API call. If the userasks about standings, the agent might query the appropriate table in the database. For text-basedquestions, it may call a search function over ingested data. The agent orchestrates this entireprocess, ensuring the LLM makes calls to the right resources dynamically.Generative AI Tools and Context:This is a standard architecture for integrating multiple functionalities into a system where each queryrequires different actions. The core design in option B is efficient because it keeps the systemmodular and dynamic by leveraging tools rather than overloading the LLM with static information ina system prompt (like option D).Why Other Options Are Less Suitable:A (RAG Architecture): While relevant, simply ingesting PDFs into a vector store only helps with textbasedretrieval. It wouldnt help with API lookups or table queries.C (Conditional Logic with RAG/API/TABLE): Although this approach works, it relies heavily on manualtext parsing and might introduce complexity when scaling the system.D (System Prompt with Event Dates and Standings): Hardcoding dates and table information into asystem prompt isnt scalable. As the standings or events change, the system would need constantupdating, making it inefficient.By bundling multiple tools into a single agent-based system (as in option B), the Generative AIEngineer can best handle the diverse requirements of this system.
Question # 24
A Generative AI Engineer has been asked to build an LLM-based question-answering application. Theapplication should take into account new documents that are frequently published. The engineerwants to build this application with the least cost and least development effort and have it operate atthe lowest cost possible.Which combination of chaining components and configuration meets these requirements?
A. For the application a prompt, a retriever, and an LLM are required. The retriever output is insertedinto the prompt which is given to the LLM to generate answers. B. The LLM needs to be frequently with the new documents in order to provide most up-to-dateanswers. C. For the question-answering application, prompt engineering and an LLM are required to generateanswers. D. For the application a prompt, an agent and a fine-tuned LLM are required. The agent is used by theLLM to retrieve relevant content that is inserted into the prompt which is given to the LLM togenerate answers.
Answer: A Explanation:ï‚? Problem Context: The task is to build an LLM-based question-answering application that integratesnew documents frequently with minimal costs and development efforts.ï‚? Explanation of Options:Option A: Utilizes a prompt and a retriever, with the retriever output being fed into the LLM. Thissetup is efficient because it dynamically updates the data pool via the retriever, allowing the LLM toprovide up-to-date answers based on the latest documents without needing to frequently retrain themodel. This method offers a balance of cost-effectiveness and functionality.Option B: Requires frequent retraining of the LLM, which is costly and labor-intensive.Option C: Only involves prompt engineering and an LLM, which may not adequately handle therequirement for incorporating new documents unless its part of an ongoing retraining or updatingmechanism, which would increase costs.Option D: Involves an agent and a fine-tuned LLM, which could be overkill and lead to higherdevelopment and operational costs.Option A is the most suitable as it provides a cost-effective, minimal development approach whileensuring the application remains up-to-date with new information.
Question # 25
A Generative Al Engineer is tasked with developing a RAG application that will help a small internalgroup of experts at their company answer specific questions, augmented by an internal knowledgebase. They want the best possible quality in the answers, and neither latency nor throughput is ahuge concern given that the user group is small and theyre willing to wait for the best answer. Thetopics are sensitive in nature and the data is highly confidential and so, due to regulatoryrequirements, none of the information is allowed to be transmitted to third parties.Which model meets all the Generative Al Engineers needs in this situation?
A. Dolly 1.5B B. OpenAI GPT-4 C. BGE-large D. Llama2-70B
Answer: C Explanation:ï‚? Problem Context: The Generative AI Engineer needs a model for a Retrieval-Augmented Generation(RAG) application that provides high-quality answers, where latency and throughput are not majorconcerns. The key factors are confidentiality and sensitivity of the data, as well as the requirementfor all processing to be confined to internal resources without external data transmission.ï‚? Explanation of Options:Option A: Dolly 1.5B: This model does not typically support RAG applications as it's more focused onimage generation tasks.Option B: OpenAI GPT-4: While GPT-4 is powerful for generating responses, its standard deploymentinvolves cloud-based processing, which could violate the confidentiality requirements due toexternal data transmission.Option C: BGE-large: The BGE (Big Green Engine) large model is a suitable choice if it is configured tooperate on-premises or within a secure internal environment that meets regulatory requirements.Assuming this setup, BGE-large can provide high-quality answers while ensuring that data is nottransmitted to third parties, thus aligning with the project's sensitivity and confidentiality needs.Option D: Llama2-70B: Similar to GPT-4, unless specifically set up for on-premises use, it generallyrelies on cloud-based services, which might risk confidential data exposure.Given the sensitivity and confidentiality concerns, BGE-large is assumed to be configurable for secureinternal use, making it the optimal choice for this scenario.
Join the Conversation
Be part of the conversation — share your thoughts, reply to others, and contribute your experience.
Sun Hao
Some scenario questions about LLM fine-tuning and AI deployment workflows were interesting.
I started preparing for the Databricks Generative AI Engineer Associate exam using practice questions. Generative AI and LLM concepts are quite detailed.
Sun Hao
Some scenario questions about LLM fine-tuning and AI deployment workflows were interesting.
Frederik Klein
Those usually test Databricks generative AI engineering certification and enterprise AI deployment concepts.