欢迎来到本系列的第三部分,也是最后一部分。在第 2 部分中,我们创建了后端并将其与 Strapi 连接,以帮助保存我们的会议和转录。在本系列的这一部分中,我们将结合使用 ChatGPT 和 Strapi,只需单击按钮即可深入了解转录文本。我们还将研究一些测试以及如何将应用程序部署到 Strapi 云。
您可以在下面找到本系列的大纲:
我们需要 Strapi CMS 中的自定义端点来与 ChatGPT 连接,因此导航到终端,将目录更改为 Strapi-transcribe-api,然后运行以下命令:
yarn strapi generate
这样做将开始生成自定义 API 的过程。选择 API 选项,将其命名为 transcribe-insight-gpt,然后在询问我们是否是插件时选择 “否”。
在 src 目录中,如果我们在代码编辑器中检查 api 目录,我们应该会看到新创建的 transcribe-insight-gpt API,其中包含 routes、controllers、和服务目录。
让我们通过取消每个文件中的代码注释、重新启动服务器并导航到管理仪表板来检查它是否有效。我们希望公开访问此路线,因此单击 设置 >用户和权限插件>角色>公开,然后向下滚动到在transcribe-insight-gpt API上选择全部以将权限公开,然后单击右上角的保存。
如果我们在浏览器中输入以下内容并单击 Enter,我们应该会收到一条 “ok” 消息。
http://localhost:1337/api/transcribe-insight-gpt
我们已经确认 API 端点正在工作,让我们先将其连接到 OpenAI,安装 OpenAI 包,导航到路由目录,然后在终端中运行以下命令
yarn add openai
然后,在 .env 文件中,将 API 密钥添加到 OPENAI 环境变量中:
OPENAI=<OpenAI api key here>
现在,在transcribe-insight-gpt目录下,将routes目录中的代码更改为以下内容:
module.exports = { routes: [ { method: "POST", path: "/transcribe-insight-gpt/exampleAction", handler: "transcribe-insight-gpt.exampleAction", config: { policies: [], middlewares: [], }, }, ], };
将控制器目录中的代码更改为以下内容:
"use strict"; module.exports = { exampleAction: async (ctx) => { try { const response = await strapi .service("api::transcribe-insight-gpt.transcribe-insight-gpt") .insightService(ctx); ctx.body = { data: response }; } catch (err) { console.log(err.message); throw new Error(err.message); } }, };
services目录中的代码如下:
"use strict"; const { OpenAI } = require("openai"); const openai = new OpenAI({ apiKey: process.env.OPENAI, }); /** * transcribe-insight-gpt service */ module.exports = ({ strapi }) => ({ insightService: async (ctx) => { try { const input = ctx.request.body.data?.input; const operation = ctx.request.body.data?.operation; if (operation === "analysis") { const analysisResult = await gptAnalysis(input); return { message: analysisResult, }; } else if (operation === "answer") { const answerResult = await gptAnswer(input); return { message: answerResult, }; } else { return { error: "Invalid operation specified" }; } } catch (err) { ctx.body = err; } }, }); async function gptAnalysis(input) { const analysisPrompt = "Analyse the following text and give me a brief overview of what it means:"; const completion = await openai.chat.completions.create({ messages: [{ role: "user", content: `${analysisPrompt} ${input}` }], model: "gpt-3.5-turbo", }); const analysis = completion.choices[0].message.content; return analysis; } async function gptAnswer(input) { const answerPrompt = "Analyse the following text and give me an answer to the question posed: "; const completion = await openai.chat.completions.create({ messages: [{ role: "user", content: `${answerPrompt} ${input}` }], model: "gpt-3.5-turbo", }); const answer = completion.choices[0].message.content; return answer; }
在这里,我们将两个参数传递给我们的 API:输入文本(将是我们的转录)和操作(根据我们希望它执行什么操作,这将是分析或答案)。每次操作都会有不同的ChatGPT提示。
我们可以通过在终端中粘贴以下代码来检查与 POST 路由的连接:
curl -X POST \ http://localhost:1337/api/transcribe-insight-gpt/exampleAction \ -H 'Content-Type: application/json' \ -d '{ "data": { "input": "Comparatively, four-dimensional space has an extra coordinate axis, orthogonal to the other three, which is usually labeled w. To describe the two additional cardinal directions", "operation": "analysis" } }'
并且要检查答案操作,您可以使用以下命令:
curl -X POST \ http://localhost:1337/api/transcribe-insight-gpt/exampleAction \ -H 'Content-Type: application/json' \ -d '{ "data": { "input": "I speak without a mouth and hear without ears. I have no body, but I come alive with the wind. What am I?", "operation": "answer" } }'
太棒了。现在我们在 Strapi API 路径中拥有了分析和应答功能,我们需要将其连接到我们的前端代码,并确保我们可以保存这些信息以用于我们的会议和转录。
为了保持清晰的关注点分离,让我们为应用程序的分析功能创建一个单独的 API 文件。
在 transcribe-frontend 的 api 目录下,创建一个名为analysis.js 的新文件并粘贴以下代码:
const baseUrl = 'http://localhost:1337'; const url = `${baseUrl}/api/transcribe-insight-gpt/exampleAction`; export async function callInsightGpt(operation, input) { console.log('operation - ', operation); const payload = { data: { input: input, operation: operation, }, }; try { const response = await fetch(url, { method: 'POST', headers: { 'Content-Type': 'application/json', }, body: JSON.stringify(payload), }); const data = await response.json(); return data; } catch (error) { console.error('Error:', error); } }
上面的代码是一个 POST 请求,用于调用 Insight API 并从 ChatGPT 返回分析结果。
让我们添加一种通过分析和答案来更新转录的方法。将以下代码粘贴到transcriptions.js 文件中。
export async function updateTranscription( updatedTranscription, transcriptionId ) { const updateURL = `${url}/${transcriptionId}`; const payload = { data: updatedTranscription, }; try { const res = await fetch(updateURL, { method: 'PUT', headers: { 'Content-Type': 'application/json', }, body: JSON.stringify(payload), }); return await res.json(); } catch (error) { console.error('Error updating meeting:', error); throw error; } }
上面的代码是一个 PUT 请求,用于处理每个转录的分析或答案字段的更新。
现在,让我们创建一个可以使用此方法的钩子。在 hooks 目录下创建一个名为 useInsightGpt 的文件,并粘贴以下代码:
import { useState } from 'react'; import { callInsightGpt } from '../api/analysis'; import { updateMeeting } from '../api/meetings'; import { updateTranscription } from '../api/transcriptions'; export const useInsightGpt = () => { const [loadingAnalysis, setLoading] = useState(false); const [transcriptionIdLoading, setTranscriptionIdLoading] = useState(''); const [analysisError, setError] = useState(null); const getAndSaveTranscriptionAnalysis = async ( operation, input, transcriptionId ) => { try { setTranscriptionIdLoading(transcriptionId); // Get insight analysis / answer const { data } = await callInsightGpt(operation, input); // Use transcriptionId to save it to the transcription const updateTranscriptionDetails = operation === 'analysis' ? { analysis: data.message } : { answer: data.message }; await updateTranscription(updateTranscriptionDetails, transcriptionId); setTranscriptionIdLoading(''); } catch (e) { setTranscriptionIdLoading(''); setError('Error getting analysis', e); } }; const getAndSaveOverviewAnalysis = async (operation, input, meetingId) => { try { setLoading(true); // Get overview insight const { data: { message }, } = await callInsightGpt(operation, input); // Use meetingId to save it to the meeting const updateMeetingDetails = { overview: message }; await updateMeeting(updateMeetingDetails, meetingId); setLoading(false); } catch (e) { setLoading(false); setError('Error getting overview', e); } }; return { loadingAnalysis, transcriptionIdLoading, analysisError, getAndSaveTranscriptionAnalysis, getAndSaveOverviewAnalysis, }; };
此挂钩处理在会议结束时获取和保存会议概述的逻辑。它还负责获取转录的分析或答案并保存它们。它会跟踪我们请求分析的转录,以便我们可以显示特定的加载状态。
Import the functionality above into the TranscribeContainer and use it. Paste the following updated code into TranscribeContainer.jsx
import React, { useState, useEffect } from "react"; import styles from "../styles/Transcribe.module.css"; import { useAudioRecorder } from "../hooks/useAudioRecorder"; import RecordingControls from "../components/transcription/RecordingControls"; import TranscribedText from "../components/transcription/TranscribedText"; import { useRouter } from "next/router"; import { useMeetings } from "../hooks/useMeetings"; import { useInsightGpt } from "../hooks/useInsightGpt"; import { createNewTranscription } from "../api/transcriptions"; const TranscribeContainer = ({ streaming = true, timeSlice = 1000 }) => { const router = useRouter(); const [meetingId, setMeetingId] = useState(null); const [meetingTitle, setMeetingTitle] = useState(""); const { getMeetingDetails, saveTranscriptionToMeeting, updateMeetingDetails, loading, error, meetingDetails, } = useMeetings(); const { loadingAnalysis, transcriptionIdLoading, analysisError, getAndSaveTranscriptionAnalysis, getAndSaveOverviewAnalysis, } = useInsightGpt(); const apiKey = process.env.NEXT_PUBLIC_OPENAI_API_KEY; const whisperApiEndpoint = "https://api.openai.com/v1/audio/"; const { recording, transcribed, handleStartRecording, handleStopRecording, setTranscribed, } = useAudioRecorder(streaming, timeSlice, apiKey, whisperApiEndpoint); const { ended } = meetingDetails; const transcribedHistory = meetingDetails?.transcribed_chunks?.data; useEffect(() => { const fetchDetails = async () => { if (router.isReady) { const { meetingId } = router.query; if (meetingId) { try { await getMeetingDetails(meetingId); setMeetingId(meetingId); } catch (err) { console.log("Error getting meeting details - ", err); } } } }; fetchDetails(); }, [router.isReady, router.query]); useEffect(() => { setMeetingTitle(meetingDetails.title); }, [meetingDetails]); const handleGetAnalysis = async (input, transcriptionId) => { await getAndSaveTranscriptionAnalysis("analysis", input, transcriptionId); // re-fetch meeting details await getMeetingDetails(meetingId); }; const handleGetAnswer = async (input, transcriptionId) => { await getAndSaveTranscriptionAnalysis("answer", input, transcriptionId); // re-fetch meeting details await getMeetingDetails(meetingId); }; const handleStopMeeting = async () => { // provide meeting overview and save it // getMeetingOverview(transcribed_chunks) await updateMeetingDetails( { title: meetingTitle, ended: true, }, meetingId, ); // re-fetch meeting details await getMeetingDetails(meetingId); setTranscribed(""); }; const stopAndSaveTranscription = async () => { // save transcription first let { data: { id: transcriptionId }, } = await createNewTranscription(transcribed); // make a call to save the transcription chunk here await saveTranscriptionToMeeting(meetingId, meetingTitle, transcriptionId); // re-fetch current meeting which should have updated transcriptions await getMeetingDetails(meetingId); // Stop and clear the current transcription as it's now saved await handleStopRecording(); }; const handleGoBack = () => { router.back(); }; if (loading) return <p>Loading...</p>; return ( <div style={{ margin: "20px" }}> {ended && ( <button onClick={handleGoBack} className={styles.goBackButton}> Go Back </button> )} {!ended && ( <button className={styles["end-meeting-button"]} onClick={handleStopMeeting} > End Meeting </button> )} {ended ? ( <p className={styles.title}>{meetingTitle}</p> ) : ( <input onChange={(e) => setMeetingTitle(e.target.value)} value={meetingTitle} type="text" placeholder="Meeting title here..." className={styles["custom-input"]} /> )} <div> {!ended && ( <div> <RecordingControls handleStartRecording={handleStartRecording} handleStopRecording={stopAndSaveTranscription} /> {recording ? ( <p className={styles["primary-text"]}>Recording</p> ) : ( <p>Not recording</p> )} </div> )} {/*Current transcription*/} {transcribed && <h1>Current transcription</h1>} <TranscribedText transcribed={transcribed} current={true} /> {/*Transcribed history*/} <h1>History</h1> {transcribedHistory ?.slice() .reverse() .map((val, i) => { const transcribedChunk = val.attributes; const text = transcribedChunk.text; const transcriptionId = val.id; return ( <TranscribedText key={transcriptionId} transcribed={text} answer={transcribedChunk.answer} analysis={transcribedChunk.analysis} handleGetAnalysis={() => handleGetAnalysis(text, transcriptionId) } handleGetAnswer={() => handleGetAnswer(text, transcriptionId)} loading={transcriptionIdLoading === transcriptionId} /> ); })} </div> </div> ); }; export default TranscribeContainer;
Here, depending on your need, we use the useInsightGpt hook to get the analysis or answer. We also display a loading indicator beside the transcribed text.
Paste the following code into TranscribedText.jsx to update the UI accordingly.
import styles from '../../styles/Transcribe.module.css'; function TranscribedText({ transcribed, answer, analysis, handleGetAnalysis, handleGetAnswer, loading, current, }) { return ( <div className={styles['transcribed-text-container']}> <div className={styles['speech-bubble-container']}> {transcribed && ( <div className={styles['speech-bubble']}> <div className={styles['speech-pointer']}></div> <div className={styles['speech-text-question']}>{transcribed}</div> {!current && ( <div className={styles['button-container']}> <button className={styles['primary-button-analysis']} onClick={handleGetAnalysis} > Get analysis </button> <button className={styles['primary-button-answer']} onClick={handleGetAnswer} > Get answer </button> </div> )} </div> )} </div> <div> <div className={styles['speech-bubble-container']}> {loading && ( <div className={styles['analysis-bubble']}> <div className={styles['analysis-pointer']}></div> <div className={styles['speech-text-answer']}>Loading...</div> </div> )} {analysis && ( <div className={styles['analysis-bubble']}> <div className={styles['analysis-pointer']}></div> <p style={{ margin: 0 }}>Analysis</p> <div className={styles['speech-text-answer']}>{analysis}</div> </div> )} </div> <div className={styles['speech-bubble-container']}> {answer && ( <div className={styles['speech-bubble-right']}> <div className={styles['speech-pointer-right']}></div> <p style={{ margin: 0 }}>Answer</p> <div className={styles['speech-text-answer']}>{answer}</div> </div> )} </div> </div> </div> ); } export default TranscribedText;
We can now request analysis and get answers to questions in real-time straight after they have been transcribed.
When the user ends the meeting, we want to provide an overview of everything discussed. Let's add this functionality to the TranscribeContainer component.
In the function handleStopMeeting we can use the method getAndSaveOverviewAnalysis from the useInsightGpt hook:
const handleStopMeeting = async () => { // provide meeting overview and save it const transcribedHistoryText = transcribedHistory .map((val) => `transcribed_chunk: ${val.attributes.text}`) .join(', '); await getAndSaveOverviewAnalysis( 'analysis', transcribedHistoryText, meetingId ); await updateMeetingDetails( { title: meetingTitle, ended: true, }, meetingId ); // re-fetch meeting details await getMeetingDetails(meetingId); setTranscribed(''); };
Here, we are joining all of the transcribed chunks from the meeting and then sending them to our ChatGPT API for analysis, where they will be saved for our meeting.
Now, let's display the overview once it has been loaded. Add the following code above the RecordingControls:
{loadingAnalysis && <p>Loading Overview...</p>} {overview && ( <div> <h1>Overview</h1> <p>{overview}</p> </div> )}
Then, destructure the overview from the meeting details by adding the following line below our hook declarations:
const { ended, overview } = meetingDetails;
To summarise, we listen to the loading indicator from useInsightGpt and check if overview is present from the meeting; if it is, we display it.
We have a couple of errors that could be caused by one of our hooks; let's create a component to handle them.
Create a file called ErrorToast.js under the components directory:
import { useEffect, useState } from 'react'; const ErrorToast = ({ message, duration }) => { const [visible, setVisible] = useState(true); useEffect(() => { const timer = setTimeout(() => { setVisible(false); }, duration); return () => clearTimeout(timer); }, [duration]); if (!visible) return null; return <div className="toast">{message}</div>; }; export default ErrorToast;
And add the following css code to globals.css under the style directory:
.toast { position: fixed; top: 20px; left: 50%; transform: translateX(-50%); background-color: rgba(255, 0, 0, 0.8); color: white; padding: 16px; border-radius: 8px; box-shadow: 0 4px 8px rgba(0, 0, 0, 0.1); z-index: 1000; transition: opacity 0.5s ease-out; opacity: 1; display: flex; align-items: center; justify-content: center; text-align: center; } .toast-hide { opacity: 0; }
Now, we can use this error component in TranscribeContainer; whenever we encounter an unexpected error from the API, we will show this error toast briefly to notify the user that something went wrong.
Import the ErrorToast at the top of the file and then paste the following code above the Go Back button in the return statement of our component:
{error || analysisError ? ( <ErrorToast message={error || analysisError} duration={5000} /> ) : null}
Now, let's add a test to ensure our hooks are working as we expect them to and to alert us to any breaking changes in the code that might be introduced later. First, add the packages below so we can use jest in our project.
yarn add -D jest jest-environment-jsdom @testing-library/react @testing-library/jest-dom @testing-library/react-hooks
Then create a jest.config.js file in the route of the frontend project and add the following code:
const nextJest = require('next/jest'); const createJestConfig = nextJest({ dir: './', }); const customJestConfig = { moduleDirectories: ['node_modules', '<rootDir>/'], testEnvironment: 'jest-environment-jsdom', }; module.exports = createJestConfig(customJestConfig);
This just sets up Jest ready to be used in Next.js.
Create a test directory and an index.test.js file with the following code:
import { renderHook, act } from '@testing-library/react-hooks'; import { useInsightGpt } from '../hooks/useInsightGpt'; import { callInsightGpt } from '../api/analysis'; import { updateMeeting } from '../api/meetings'; import { updateTranscription } from '../api/transcriptions'; jest.mock('../api/analysis'); jest.mock('../api/meetings'); jest.mock('../api/transcriptions'); describe('useInsightGpt', () => { beforeEach(() => { jest.clearAllMocks(); }); it('should handle transcription analysis successfully', async () => { const mockData = { data: { message: 'Test analysis message' } }; callInsightGpt.mockResolvedValueOnce(mockData); updateTranscription.mockResolvedValueOnce({}); const { result } = renderHook(() => useInsightGpt()); await act(async () => { await result.current.getAndSaveTranscriptionAnalysis( 'analysis', 'input', 'transcriptionId' ); }); expect(callInsightGpt).toHaveBeenCalledWith('analysis', 'input'); expect(updateTranscription).toHaveBeenCalledWith( { analysis: 'Test analysis message' }, 'transcriptionId' ); expect(result.current.transcriptionIdLoading).toBe(''); expect(result.current.analysisError).toBe(null); }); it('should handle overview analysis successfully', async () => { const mockData = { data: { message: 'Test overview message' } }; callInsightGpt.mockResolvedValueOnce(mockData); updateMeeting.mockResolvedValueOnce({}); const { result } = renderHook(() => useInsightGpt()); await act(async () => { await result.current.getAndSaveOverviewAnalysis( 'overview', 'input', 'meetingId' ); }); expect(callInsightGpt).toHaveBeenCalledWith('overview', 'input'); expect(updateMeeting).toHaveBeenCalledWith( { overview: 'Test overview message' }, 'meetingId' ); expect(result.current.loadingAnalysis).toBe(false); expect(result.current.analysisError).toBe(null); }); it('should handle errors in transcription analysis', async () => { const mockError = new Error('Test error'); callInsightGpt.mockRejectedValueOnce(mockError); const { result } = renderHook(() => useInsightGpt()); await act(async () => { await result.current.getAndSaveTranscriptionAnalysis( 'analysis', 'input', 'transcriptionId' ); }); expect(result.current.transcriptionIdLoading).toBe(''); expect(result.current.analysisError).toBe( 'Error getting analysis', mockError ); }); it('should handle errors in overview analysis', async () => { const mockError = new Error('Test error'); callInsightGpt.mockRejectedValueOnce(mockError); const { result } = renderHook(() => useInsightGpt()); await act(async () => { await result.current.getAndSaveOverviewAnalysis( 'overview', 'input', 'meetingId' ); }); expect(result.current.loadingAnalysis).toBe(false); expect(result.current.analysisError).toBe( 'Error getting overview', mockError ); }); });
Because the hooks use our Strapi API, we need a way to replace the data we're getting back from the API calls. We're using jest.mock to intercept the APIs and send back mock data. This way, we can test our hooks' internal logic without calling the API.
In the first two tests, we mock the API call and return some data, then render our hook and call the correct function. We then check if the correct functions have been called with the correct data from inside the hook. The last two tests just test that errors are handled correctly.
Add the following under scripts in the package.json file:
"test": "jest --watch"
Now open the terminal, navigate to the route directory of the frontend project, and run the following command to check if the tests are passing:
yarn test
You should see a success message like the one below:
As an optional challenge, let's see if you can apply what we did with testing useInsightGpt to testing the other hooks.
Here is what our application looks like.
Finally, we have the finished application up and running correctly with some tests. The time has come to deploy our project to Strapi cloud.
First, navigate to Strapi and click on "cloud" at the top right.
Connect with GitHub.
From the dashboard, click on Create project.
Choose your GitHub account and the correct repo, fill out the display name, and choose the region.
Now, if you have the same file structure as me, which you should do if you've been following along, then you will just need to add the base directory, so click on Show advanced settings and enter the base directory of /strapi-transcribe-api, then you will need to add all of the environment variables that can be found in the .env file in the route of the strapi project.
Once you have added all of these, click on "create project." This will bring you to a loading screen, and then you will be redirected to the build logs; here, you can just wait for the build to finish.
Once it has finished building, you can click on Overview from the top left. This should direct you to the dashboard, where you will find the details of your deployment and the app URL under Overview on the right.
First, click on your app URL, which will open a new tab and direct you to the welcome page of your Strapi app. Then, create a new admin user, which will log you into the dashboard.
This is a new deployment, and as such, it won't have any of the data we had saved locally; it also won't have carried across the public settings we had on the API, so click on Settings>Users & Permissions Plugin>Roles>Public, expand and select all on Meeting, Transcribe-insight-gpt, and Transcribed-chunk, and then click save in the top right.
Once again, let's just check that our deployment was successful by running the below command in the terminal. Please replace https://yourDeployedUrlHere.com with the URL in the Strapi cloud dashboard.
curl -X POST \ https://yourDeployedUrlHere.com/api/transcribe-insight-gpt/exampleAction \ -H 'Content-Type: application/json' \ -d '{ "data": { "input": "I speak without a mouth and hear without ears. I have no body, but I come alive with the wind. What am I?", "operation": "answer" } }'
Now we have the API deployed and ready to use, let's deploy our frontend with Vercel.
First, we will need to change the baseUrl in our API files to link to our newly deployed Strapi instance,
Add the following variable to .env.local
NEXT_PUBLIC_STRAPI_URL="your strapi cloud url"
Now go ahead and replace the current value of baseUrl with the following in all three API files:
const baseUrl = process.env.NODE_ENV == 'production' ? process.env.NEXT_PUBLIC_STRAPI_URL : 'http://localhost:1337';
This will just check if the app is running in production. If so, it will use our deployed strap instance. If not, it will revert to localhost. Make sure to push these changes to Github.
Now navigate to Vercel and sign up if you don't already have an account.
Now, let's create a new project by continuing with GitHub.
Once you have verified your account, import the correct GitHub repo
Now we will fill out some configuration details, give the project a name, change the framework preset to Next.js, change the root directory to 'transcribe-frontend', and add the two environment variables from the .env.local file in the Next.js project.
Now click deploy and wait for it to finish. Once deployed, it should redirect you to a success page with a preview of the app.
Now click continue to the dashboard, where you can find information about the app, such as the domain and the deployment logs.
From here, you can click visit to be directed to the app's frontend deployment.
所以你就有了!您现在已经从头到尾构建了转录应用程序。我们已经研究了如何利用几种尖端技术来实现这一目标。我们使用 Strapi 进行后端 CMS 和自定义 ChatGPT 集成,展示了该技术如何快速、轻松地构建复杂的 Web 应用程序。我们还介绍了 Next.js 中的错误处理和测试的一些架构模式,最后,我们将后端部署到 Strapi 云。我希望这个系列让您大开眼界,并鼓励您将想法变为现实。
以上是使用 Strapi、ChatGPT 和 Whisper 构建转录应用程序:第 3 部分的详细内容。更多信息请关注PHP中文网其他相关文章!