engineering/uncloseai.com
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

fix countTokens returning object instead of number causing incorrect chunking

Browse source
This commit is contained in:
Russell Ballestrini 2025-07-11 20:48:40 -04:00
parent af6108e988
commit 84b06202dd
1 changed files with 2 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

3
src/translation.js
View file

@ -511,7 +511,8 @@ export async function translateHTML(htmlContent, targetLanguage, sourceUrl = nul
const availableTokens = Math.max(maxTokens - reserveTokens, 2000); // Ensure minimum
// Check if content needs chunking based on token count (on raw HTML, not preserved)
const contentTokens = await countTokens(processedContent, getSelectedModel());
const contentTokensResult = await countTokens(processedContent, getSelectedModel());
const contentTokens = typeof contentTokensResult === 'object' ? contentTokensResult.count || contentTokensResult.totalTokens || 0 : contentTokensResult;
console.log(`=== CHUNKING DECISION ===`);
console.log(`Max tokens: ${maxTokens}`);