Skip to content

Add related articles #207

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 800 commits into from
Closed

Add related articles #207

wants to merge 800 commits into from

Conversation

wkenned1
Copy link
Collaborator

@wkenned1 wkenned1 commented Dec 7, 2024

Fixes #206

@wkenned1 wkenned1 changed the title Test related articles Add related articles backend Dec 26, 2024
@wkenned1 wkenned1 changed the title Add related articles backend Add related articles Dec 31, 2024
@wkenned1 wkenned1 marked this pull request as ready for review January 24, 2025 04:56
astro.config.mjs Outdated
Comment on lines 8 to 139
}
return match ? match[1].trim() : "";
}

function parseConclusion(markdown) {
const regex = /## Conclusion\s+([\s\S]*?)(?=\n\n|$)/;
const match = markdown.match(regex);

if (match) {
console.log(match[1].trim());
}
return match ? match[1].trim() : "";
}

function buildArticleJson() {
articleFiles.forEach((value, key) => {
const article = fs.readFileSync(path.join(value), "utf8");
const { data } = matter(article);
const obj = {
slug: key,
content: `${data.title}. ${data.excerpt}. ${data.tags.join(" ")}. ${parseIntroduction(article)}`,
};
console.log(obj);
articleObjs.push(obj);
});

console.log(`Read ${articleObjs.length} articles.`);
}

async function addEmbeddedArticles() {
console.log("Vectorizing articles...");

const directory = "./src/content/articles";

readDirectory(directory, null);
buildArticleJson();

try {
const externalResponse = await fetch(
"https://related-articles.andrei-023.workers.dev/add_articles",
{
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ articles: articleObjs }),
},
);

console.log("Uploaded articles");
} catch (error) {
console.error("Error sending files:", error);
}
}

async function getArticleMatches() {
try {
const res = await fetch(
"https://related-articles.andrei-023.workers.dev/match_articles",
{
method: "GET",
headers: { "Content-Type": "application/json" },
},
);

const body = await res.json();

fs.writeFile(
"src/data/matchedArticles.json",
JSON.stringify(body),
(err) => {
if (err) {
console.error(err);
return;
}
console.log("JSON data saved to data.json");
},
);
} catch (error) {
console.error("Error sending files:", error);
}
}

// Custom Astro integration
function buildStart() {
return {
name: "my-build-start",
hooks: {
"astro:build:start": async () => {
const branch = process.env.CF_PAGES_BRANCH || "unknown";
if (branch === "main") {
await addEmbeddedArticles();
}

await getArticleMatches();

// await addEmbeddedArticles();
// await getArticleMatches();
},
},
};
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Move to a file

@@ -8,14 +8,19 @@ export async function getStaticPaths() {

return articles.map((article) => ({
params: { slug: article.slug },
props: { article },
props: { article, articles },
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
props: { article, articles },
props: { article, relatedArticles },

}));
}

const { article } = Astro.props;
const { article, articles } = Astro.props;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const { article, articles } = Astro.props;
const { article, relatedArticles } = Astro.props;

<ArticleLayout {article} {headings} minutesRead={getReadingTime(article.body)}>
<ArticleLayout
{article}
{articles}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
{articles}
{relatedArticles}

headings: MarkdownHeading[];
minutesRead?: number;
}

const { article, headings, minutesRead } = Astro.props;
const { article, articles, headings, minutesRead } = Astro.props;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const { article, articles, headings, minutesRead } = Astro.props;
const { article, relatedArticles, headings, minutesRead } = Astro.props;

import FloatingActionMenuHelper from "../_components/FloatingActionMenuHelper";

interface Props {
article: CollectionEntry<"articles">;
articles: CollectionEntry<"articles">[];
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
articles: CollectionEntry<"articles">[];
relatedArticles: CollectionEntry<"articles">[];

astro.config.mjs Outdated
}

function parseConclusion(markdown) {
const regex = /## Conclusion\s+([\s\S]*?)(?=\n\n|$)/;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ending logic here should consider multiple paragraphs or a new h2 (such as references)

Comment on lines 68 to 83

// Ignore type errors when matchedArticles is blank
// @ts-ignore
const matches = matchedArticles.all_matches.filter(
// @ts-ignore
(match) => match.source_slug === article.slug,
);
// @ts-ignore
matches.sort((a, b) => b.similarity - a.similarity);

// @ts-ignore
const relatedArticles: CollectionEntry<"articles">[] = matches.map((match) =>
// @ts-ignore
(articles as CollectionEntry<"articles">[]).find((article) => article.slug === match.matched_slug),
);

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parse with zod

@@ -5,6 +5,7 @@ import defaultTheme from "tailwindcss/defaultTheme";

/** @type {import('tailwindcss').Config} */
export default {
darkMode: ["class"],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See what you actually need in this file

@wkenned1 wkenned1 deleted the wkenned1/issue206 branch April 6, 2025 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add related articles backend
4 participants