Talk with Aptos Whitepaper | Using MoveSpace 0x01
Thanks for the open-source framework Embedbase.
Homepage of MoveSpace:
https://movespace.xyz
1 Vectorization of Markdown documents
Cutting and processing raw data are the preliminary steps for vector database generation.
We design the granularity of vectors based on actual scenarios. Taking text metadata as an example, the granularity can be “word”, “sentence” or “paragraph”. In this example, we insert vectors using paragraph units. At the same time, we annotate paragraph data through comments in Makrdown.
https://github.com/NonceGeek/whitepaper-talker/blob/main/pages/index.md
Example of paragraph data annotation through comments:
## Abstract
The rise of blockchains as a new Internet infrastructure has led to developers deploying tens of thousands of decentralized applications at rapidly growing rates. Unfortunately, usage blockchain is not yet ubiquitous due to frequent outages, high costs, low throughput limits, and numerous security concerns. To enable mass adoption in the web3 era, blockchain infrastructure needs to follow the path of cloud infrastructure as a trusted, scalable, cost-efficient, and continually improving platform for building widely-used applications.
<!-- 1. Blockchains\n2. Decentralized applications\n3. Outages\n4. High costs\n5. Security concerns -->Markdown document processing code is currently implemented using Elixir:
https://github.com/NonceGeek/movespace_db/blob/main/lib/chat_programming/cli.ex
The implementation of python and js can be implemented by yourself according to your needs~
2 Use Nextra to generate documents
Create a new Repo via Nextra’s Template.
Then modify the project name in package.json:
{
"name": "aptos-whitepaper",
"version": "0.0.1",
"description": "the Whitepaper of Aptos",
...
}Modify the document content in pages, supporting md and mdx:
3 Add vector database retrieval capabilities to Nextra
3.1 Add search.ts module
Create pages/search.ts.
const url = "https://api.embedbase.xyz";
const vaultId = "aptos-whitepaper-handled";
const apiKey = process.env.NEXT_PUBLIC_EMBEDBASE_API_KEY;
const do_search = async (query: string) => {
return fetch(url + "/v1/" + vaultId + "/search", {
method: "POST",
headers: {
Authorization: "Bearer " + apiKey,
"Content-Type": "application/json"
},
body: JSON.stringify({
query: query
})
}).then(response => response.json());
};
export default async function search(question: string) {
const searchResponse = await do_search(question);
return searchResponse
}Get the search results by calling the /v1/dataset_name/search interface.
The function exposed to the outside world is search.
Here we write dataset_name as const. Of course, a more elegant way is to write it in an environment variable.
3.2 Transform the original Search function
Modify theme.config.tsx to transform the original Search function.
Add recommended questions list:
const questions = [
'What is Aptos?',
'How could I use Aptos CLI?'
]Add Markdown component:
const MarkdownComponents: object = {
code({ node, inline, className, ...props }) {const match = /language-(\w+)/.exec(className || '')
const hasMeta = node?.data?.meta
const applyHighlights: object = (applyHighlights: number) => {
if (hasMeta) {
const RE = /{([\d,-]+)}/
const metadata = node.data.meta?.replace(/\s/g, '')
const strlineNumbers = RE?.test(metadata)
? RE?.exec(metadata)[1]
: '0'
const highlightLines = rangeParser(strlineNumbers)
const highlight = highlightLines
const data: string = highlight.includes(applyHighlights)
? 'highlight'
: null
return {data}
} else {
return {}
}
}
return match ? (
<SyntaxHighlighter
children={""}
style={oneDark}
// language={match[1]}
PreTag="div"
className="codeStyle"
// showLineNumbers={true}
wrapLines={hasMeta ? true : false}
useInlineStyles={true}
lineProps={applyHighlights}
{...props}
/>
) : (
<code className={className} {...props} />
)
},
}Add the SearchBar component:
const EmbedbaseSearchBar = ({ value, onChange, autoFocus, placeholder, onClick }: EmbedbaseSearchBarProps) => {
return (
<input
autoFocus={autoFocus || false}
placeholder={placeholder || "Search..."}
onClick={onClick}
type="text"
value={value}
onChange={onChange}
// border around with smooth corners, a magnifier icon on the left,
// the search bar taking up the rest of the space
// focused on load
style={{ width: '100%', padding: '0.5rem', border: '1px solid #e5e7eb', borderRadius: '0.5rem', outline: 'none' }}
/>
);
};The qa function provides search capabilities. In this function we call the functions exposed in search.ts:
const qa = async (e: any) => {
e.preventDefault();
setLoading(true);
setOutput("");
const searchResp = await search(prompt);
// get pure.
let similarities_handled = [];
for(let i in searchResp.similarities){
similarities_handled.push(searchResp.similarities[i].data);
}
console.log(JSON.stringify(similarities_handled));
let prompt_finally = "Here are the contents:\n\n"
for(let i in similarities_handled){
prompt_finally = prompt_finally + "* " + similarities_handled[i] + "\n\n";
}
prompt_finally += "So the question is: " + prompt;
setOutput(prompt_finally);
setLoading(false);
};Finally we provided the front-end display of SearchModal:
<div>
{/* on click, open modal */}
<EmbedbaseSearchBar onClick={() => setOpen(true)} placeholder="Ask a question..." />
<Modal open={open} onClose={onClose}>
<form onSubmit={qa} className="nx-flex nx-gap-3">
<EmbedbaseSearchBar value={prompt} onChange={(e) => setPrompt(e.target.value)} autoFocus />
<button
className="nx-rounded-full nx-bg-sky-300 nx-py-2 nx-px-4 nx-text-sm nx-font-semibold nx-text-slate-900 nx-hover:nx-bg- sky-200 nx-focus:outline-none focus-visible:outline-2 focus-visible:nx-outline-offset-2 nx-focus-visible:nx-outline-sky-300/50 nx-active:bg-sky -500nx-max-w-max"
type="submit"
>
Ask
</button>
</form>
{/* row oriented, centered, with a gap of 3 */}
<div className="nx-flex nx-gap-3 nx-py-4 nx-min-h-40 nx-flex-col">
{!loading && output.length < 1 && (
<div className="nx-text-gray-400 nx-text-sm nx-font-semibold">
Your result will appear here</div>
)}
{loading && (
<div className="nx-flex nx-items-center nx-justify-center">
<span>Loading...</span>
<div
style={{
width: "1rem",
height: "1rem",
border: "1px solid #e5e7eb",
borderRadius: "50%",
borderTopColor: "black",animation: "spin 1s linear infinite",
}}
></div>
</div>
)}
{!loading && output.length > 0 && (
<div>
<ReactMarkdown
components={MarkdownComponents}
>{output}</ReactMarkdown>
<a href="https://chat.openai.com/" target="_blank">
<button
className="nx-rounded-full nx-bg-sky-300 nx-py-2 nx-px-4 nx-text-sm nx-font-semibold nx-text-slate-900 nx-hover:nx-bg-sky-200 nx-focus:outline-none focus-visible:outline-2 focus-visible:nx-outline-offset-2 nx-focus-visible:nx-outline-sky-300/50 nx-active:bg-sky-500 nx-max-w-max"
>
Visit ChatGPT to Ask!
</button>
</a>
</div>
)}
</div>
<div
style={{
borderTop: "1px solid #e5e7eb",
marginTop: "1rem",
}}
>
{/* try one of these samples */}
<div className="nx-mt-2">Try one of these samples:</div>
<div
style={{
cursor: "pointer",
fontSize: "0.9rem",
display: "flex",
alignItems: "center",
gap: "0.5rem",
fontWeight: 600,
}}
// examples as a list of bullets
className="nx-flex-row"
>
<ul>
{questions.map((q) => (
// in row orientation, centered, with a gap of 3
<li key={q} onClick={() => setPrompt(q)}>
* {q}
</li>
))}
</ul>
</div>
<div
style={{
display: "flex",
justifyContent: "space-between",
alignItems: "center",
padding: "0.5rem 0",
fontSize: "0.75rem",
}}
>
<div
style={{
display: "flex",
alignItems: "center",
gap: "0.25rem",
paddingTop: "0.5rem",
paddingBottom: "0.25rem",
}}
>
<a href="https://movespace.xyz" className="underline">
Powered by MoveSpace.xyz & Embedbase
</a>
</div>
</div>
</div>
</Modal>
</div>Take a look at the final app:
4 Vector database retrieval capability back-end
As an example, we call embedbase directly on the front end, but the problem with this is that embedbase-key will be exposed. A good practice is to write a backend and interact with embedbase through the backend. This is left to the readers to complete by themselves^^.
