-
Notifications
You must be signed in to change notification settings - Fork 44
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Parse JSON Notices Section #2669
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
need to run prettier
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please rerun pre-commit hooks.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2669 +/- ##
========================================
- Coverage 6.06% 5.93% -0.14%
========================================
Files 171 171
Lines 4334 4329 -5
Branches 476 474 -2
========================================
- Hits 263 257 -6
- Misses 4069 4070 +1
Partials 2 2 ☔ View full report in Codecov by Sentry. |
This encoding information needs to also be referenced somewhere in the creating JSON notices documentation. |
Add example of pulling message off the network and getting bytes out. |
261b6ab
to
3f09546
Compare
c2e6704
to
e9225b0
Compare
@dakota002 please provide your suggestions. |
app/routes/docs.client.samples.md
Outdated
# Function to validate Base64 strings | ||
def is_base64(s): | ||
try: | ||
base64.b64decode(s, validate=True) | ||
return True | ||
except Exception: | ||
return False | ||
|
||
# Output: A Base64 encoded bytestring, e.g., b'a1512dabc1b6adb3cd1b6dcb6d4c6......' | ||
with open("path/to/encoded_file.txt", 'wb') as encoded_file: | ||
encoded_file.write(encoded_string) | ||
# Validate the skymap string | ||
if not is_base64(skymap_string): | ||
print("Invalid Base64 string.") | ||
continue |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this necessary?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We can remove it, it's just a check if file wasn't corrupted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please remove.
app/routes/docs.client.samples.md
Outdated
import base64 | ||
|
||
# Set Kafka Topic and Producer Configuration | ||
TOPIC = "gcn.circulars" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't seem like the right topic.
app/routes/docs.client.samples.md
Outdated
# Connect as a Kafka consumer | ||
consumer = Consumer(client_id='fill me in', # Replace with your client ID | ||
client_secret='fill me in', # Replace with your client secret | ||
config={"message.max.bytes": 204194304}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's this for?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
app/routes/docs.client.samples.md
Outdated
|
||
producer = Producer(client_id='fill me in', # Replace with your client ID | ||
client_secret='fill me in', # Replace with your client secret | ||
config={"message.max.bytes": 204194304}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do you need this?
Note that there are merge conflicts. Please fix them. |
ffb57c3
to
d6f64a8
Compare
I was rebasing for merge conflicts, looks it got closed :( |
Still working on this branch |
This document includes how to parse, encode, decode the JSON Schema files.
Resolves: #2149
Previous branch (#2506) had 100+ commits behind, rebase was messy and touched many files.
Feedbacks need to be included:
-Provide hyperlinks to these Python methods.
-Please provide sample code for parsing Kafka records in a message consuming loop