created: 2024-12-21 08:42 tags: - daily-notes

Saturday, December 21, 2024

<< Timestamps/2024/12-December/2024-12-20-Friday|Yesterday | Timestamps/2024/12-December/2024-12-22-Sunday|Tomorrow >>

🎯 Goal

• [x] Incorporate the tag_notes Django Management Command into the Celery scheduled task so that notes are tagged after upload
• [x] Make sure that the relative path of notes is updated / check whether the relative path of the file has changed on import in the import_vault_from_s3 Django Management Command

🌟 Results

• Added the tag_notes Django Management Command to the scheduled task so that notes get tagged upon upload
• Updated the Note's relative_path attribute based on where it is in the actual S3 Bucket
• Fixed the bug where moving files to a new directory uploaded duplicate files
• Extracted summaries from notes via custom logic by creating the update_daily_note_summary Django Management Command
• Added update_daily_note_summary to the Note import pipeline scheduled task so that it goes:
  • import_obsidian_vault_from_s3,
  • tag_notes,
  • update_daily_note_summary

🌱 Next Time

• Start working with LinkedNotes of the BigBrain App to display all incoming and outgoing links on the page in a well-formatted way.
• Maybe change the position of the next and previous note links on the note_details Django Template to be at the top of the page instead of the bottom.

📝 Notes

I started today by correcting the formatted_markdown rendering property in the Note Django Model. It didn't handle the edge case for when it encounters an empty demo of [[]], meaning that it was trying to route to an empty Vault Slug of ''.

Next I wanted to add the tag_notes Django Management Command to the Celery task. This was easy enough because the run_import_obsidian_vault_from_s3 shared task was already established. Therefore all I had to do was add a call to the tag_notes command from this shared task in a similar format to the import command. I was able to validate that this worked with the Django Shell before proceeding to update paths / tags.

Next I looked at the import_obsidian_vault_from_s3 command itself and added a quick check to see if the relative path has changed. This led me to realize that on uploading my notes, files are uploaded to new locations instead of being deleted first. For instance, after creating the Regex.md file (Regex) it was automatically placed in the primary directory /Regex.md. However once I moved it to its proper folder (/Web Development/Regex.md) the original file remained in the S3 Bucket's location but there was a new file uploaded to the new location. Therefore I needed to delete that original file before uploading it. I was able to handle this with the following logic:

# Detect moved or renamed files
moved_files = []
for old_key in list(s3_metadata.keys()):
    if old_key not in local_file_metadata:
        moved_files.append(old_key)

# Delete moved files from S3
for old_key in moved_files:
try:
    s3.delete_object(Bucket=AWS_BUCKET_NAME, Key=old_key)
    self.stdout.write(self.style.SUCCESS(f"Deleted moved/renamed file: {old_key}"))
except Exception as e:
    self.stderr.write(self.style.ERROR(f"Failed to delete {old_key}: {e}"))

Now files are deleted from their old path in the S3 Bucket before being uploaded to the new one (to prevent duplicate file conflicts).

The next step in processing these notes will be a little more involved and will mean parsing the content of the notes and using some selection for the summary. In these notes that's easy, I can just split by the --- characters and look for the specific location with the special emoji I used in the results section. I can also look for the specific signature of the summary in my WGU notes as well as my old note format here (which there may not be one?). Either way I like breaking these components into separate Django Management Commands so that I can treat them as separate parts of the import process. Just because a note can't be tagged doesn't mean it can't be uploaded which doesn't mean it can't be parsed for a summary, etc.

I created the update_daily_note_summary.py Django Management Command and had it select specifically for notes within the Daily Note Tag. I was able to set the summary properly, but needed to update the way I display the markdown in my Django View. Specifically, the way that the formatted markdown data is rendered only allows for the Note.content to be appropriately translated into markdown. Therefore I broke up the logic in the Django View to handle translating the text into a Markdown-safe context, then created formatted markdown specifically for both content and summary fields. Now the summaries can be displayed neatly in the vault_notes_list:

--redacted--

Interestingly the links here aren't bolded, so the links to the other notes are are still available and working here but aren't emphasized. Not a big deal so I'll leave it the way it is. I added some custom logic for both this vault and my WGU MSDADS vault to parse for summaries. This was easier than I thought it would be, it seems like my daily note template logic has remained fairly consistent.

Then I pushed these changes to Production so that I could use these new Django Management Commands. Then I uploaded my current notes to the S3 Bucket and ran the scheduled task through the Django Shell so that my notes could be updated on my Production Database.

Notes created today

List FROM "" WHERE file.cday = date("2024-12-21") SORT file.ctime asc

Notes last touched today

List FROM "" WHERE file.mday = date("2024-12-21") SORT file.mtime asc

(Template referenced from Dann Berg, can be found here)