本地部署OpenAI最强开源项目-OpenAI Realtime Console！零延迟、多模态、实时语音交互！用函数调用实现新闻抓取和播报的功能！AI语音助手最强开源框架，打造极致用户体验 #ai

8 min readOct 7, 2024

🔥🔥🔥本篇笔记所对应的视频 https://youtu.be/NirFxc1GvR8

openai/openai-realtime-console

git clone https://github.com/openai/openai-realtime-console.git

npm install#http://localhost:3000/
git clone https://github.com/openai/openai-realtime-api-beta.gitnpm i openai/openai-realtime-api-beta --save
#uninstallrm -rf openai-realtime-consolenpm uninstall -g openai-realtime-consolenpm cache clean --force

API Key

https://platform.openai.com/api-keys

虚拟信用卡注册地址

https://bewildcard.com/i/VLM

虚拟信用卡邀请码 VLM

test

Write Python code to get the latest news.

How should I read a book?I enjoy reading social science books.economics booksSkin in the gameWhat do you think of this book?What are the thought-provoking insights in this book?Search for the latest AI news.

👉👉👉如有问题请联系我的徽信 stoeng

🔥🔥🔥本项目代码由AI超元域频道制作，观看更多大模型微调视频请访问我的频道⬇

👉👉👉我的哔哩哔哩频道

👉👉👉我的YouTube频道

👉👉👉我的开源项目 https://github.com/win4r/AISuperDomain

server.js

import express from 'express';
import cors from 'cors';
import axios from 'axios';
import dotenv from 'dotenv';

dotenv.config();const app = express();
const port = process.env.PORT || 3001;// 正确的CORS配置
app.use(cors({
  origin: 'http://localhost:3000', // 替换为你的React应用的URL
  methods: ['GET', 'POST', 'OPTIONS'],
  allowedHeaders: ['Content-Type', 'Authorization']
}));app.use(express.json());app.get('/api/news', async (req, res) => {
  try {
    const { q } = req.query;
    const apiKey = process.env.SERPAPI_API_KEY;    if (!apiKey) {
      throw new Error('SERPAPI API key is not set');
    }    const response = await axios.get('https://serpapi.com/search', {
      params: {
        engine: 'google_news',
        q,
        gl: 'us',
        hl: 'en',
        api_key: apiKey
      }
    });    const newsResults = response.data.news_results;
    res.json(newsResults);
  } catch (error) {
    console.error('Detailed error:', error);
    res.status(500).json({ error: 'Failed to fetch news', details: error.message });
  }
});app.listen(port, () => {
  console.log(`Server running on port ${port}`);
});

Tools

//新增
    async function performGoogleSearch(query: string): Promise<Array<{title: string, url: string}>> {
      console.log('Starting Google search for:', query);
      try {
        const response = await fetch(`http://localhost:3001/api/news?q=${encodeURIComponent(query)}`);
    
        if (!response.ok) {
          throw new Error(`HTTP error! status: ${response.status}`);
        }
    
        const results = await response.json();
        console.log('Search results:', results);
    
        return results.map((item: any) => ({
          title: item.title,
          url: item.link
        }));
      } catch (error) {
        console.error('Error in performGoogleSearch:', error);
        return [{
          title: 'Error',
          url: ''
        }];
      }
    }

//1
{conversationItem.type === 'function_call_output' && 
conversationItem.formatted.tool?.name === 'google_search' && (
  <div>
    <h4>Google Search Results:</h4>
    {(() => {
      let results;
      try {
        results = typeof conversationItem.formatted.output === 'string'
          ? JSON.parse(conversationItem.formatted.output)
          : conversationItem.formatted.output;
        
        if ('error' in results) {
          return <p>Error: {results.error} {results.details && `(${results.details})`}</p>;
        }

        if (!Array.isArray(results)) {
          console.error('Unexpected search results format:', results);
          return <p>Unexpected search results format.</p>;
        }
        
        return results.map((result, index) => (
          <div key={index}>
            <h5>{result.title}</h5>
            <p>{result.description}</p>
            <a href={result.url} target="_blank" rel="noopener noreferrer">
              {result.url}
            </a>
          </div>
        ));
      } catch (error) {
        console.error('Error parsing search results:', error);
        return <p>Error displaying search results.</p>;
      }
    })()}

🔥🔥🔥本期视频详细演示了本地部署OpenAI最新发布的realtime-api和Realtime-Console开源项目。

🚀突破传统语音交互瓶颈：OpenAI Realtime-Console项目详解，从本地部署到自定义功能，打造极致用户体验。

🚀主要内容：

1️⃣详细演示Realtime-Console的使用，展示其零延迟多模态交互能力。

2️⃣对比传统语音交互方式，突出Realtime-Console的优势。

3️⃣step-by-step指导本地部署Realtime-Console项目。

4️⃣详解如何获取和设置OpenAI API密钥。

5️⃣介绍使用虚拟卡充值OpenAI账户的方法，解决支付问题。

6️⃣深入剖析如何在Realtime-Console中实现Google搜索功能。

7️⃣代码解析：定义工具和函数，实现自定义功能。

8️⃣讲解如何解决跨域问题，确保项目顺利运行。