5" Commands

[MrFr1day]

Hi,

Following @gerph suggestion, I managed to get, and test most commands (as far as I know, only "add-file" is missing), I can't implement this in code for the library, but I can post what I know, and test if you implement it.

Based on my tests, I think the way their app (UsbMonitorL) works is something like this:

1- Gather the selected PC info.
2- Render a full image in BBGGRRAA format.
2a- Play the Video (if one was set as a background).
3- Send the rendered image using a "Full_Image" command.

Then loop the following:
4- Gather PC info again.
5- Render another full image.
6- Compare it with the current image, then generate the changed-pixels message.
7- Send the changed-pixels message using an "Update_Image" command.
8- Apply the changes to the image stored in the app so it can be synced with the monitor.

---

General Commands:

• Get the device name/type and firmware (and maybe initialize):
  01 ef 69 00 00 00 01 00 00 00 c5 d3

• Set brightness, L=1-255:
  7b ef 69 00 00 00 01 00 00 00 (L)

• Set options, M=StartMode(1=Default, 2=PlayImage, 3=PlayVideo), F=Album-Flip 180 (1=Enable), S=SleepAfter in minutes (0=Disabled, app range 1-10):
  7d ef 69 00 00 00 05 00 00 00 2d (M) 00 (F) (S)

• TurnOff monitor (it will change it's port number):
  83 ef 69 00 00 00 01

• Restart monitor:
  84 ef 69 00 00 00 01

---

Storage Commands:

• Query Internal/SDCard storage usage, the reply is (InternalTotal-InternalUsed-InternalFree-SDTotal-SDUsed-SDFree):
  64 ef 69 00 00 00 01

• List Images/Videos in Internal Storage (the folder structure is already created), the reply is a list of files (result:dir:file:f1.ext/f2.ext/fN+.ext/):
  65 ef 69 00 00 00 0a 00 00 00 2f 72 6f 6f 74 2f 69 6d 67 2f
65 ef 69 00 00 00 0c 00 00 00 2f 72 6f 6f 74 2f 76 69 64 65 6f 2f

• List Images/Videos in SD Card:
  65 ef 69 00 00 00 10 00 00 00 2f 6d 6e 74 2f 53 44 43 41 52 44 2f 69 6d 67 2f
65 ef 69 00 00 00 12 00 00 00 2f 6d 6e 74 2f 53 44 43 41 52 44 2f 76 69 64 65 6f 2f

• Get a video file size P=Path, the reply is the size in bytes:
  6e ef 69 00 00 00 13 00 00 00 (P)

• Delete a video file, P=Path:
  66 ef 69 00 00 00 14 00 00 00 (P)

---

Media Commands:

• Play Video, P=Path, the reply is (play_video_success):
  78 ef 69 00 00 00 13 00 00 00 (P)

• Stop Video:
  79 ef 69 00 00 00 01

• Display a full image (for an image background):
  c8 ef 69 00 17 70

• Display an initial full image overlay (for a video background):
  ca ef 69 00 17 70

• Display a changed-pixels image (for an image background), S=Image-message bytes count (2bytes), N=Serial number of the image starting from 0 (4bytes):
  cc ef 69 00 00 (S) 00 00 00 (N)

• Display a changed-pixels image (for a video background), S=Image-message bytes count (2bytes), N=Serial number of the update starting from 0 (4bytes), V=visible-pixels info bytes count (4bytes):
  cc ef 69 00 00 (S) 00 00 00 (N) 00 00 (V)

• Query monitor render status, reply is (needReSend:0|renderCnt:X), "needReSend" will be 1 if a changed-pixels message was missed, and "renderCnt" will increase with each frame rendered by the monitor it self:
  cf ef 69 00 00 00 01

---

Image Messages (for an image background):

• Full-Image message:
  Basically, list all the pixels in BBGGRRAA hex format (e.g blue: FF0000FF)

• Changed-pixels message:
  S(6bytes) C(4bytes) P(6bytes per pixel), where:

S=starting pixel position
C=Count, how many pixels
P=PixelColors in BBGGRR format

There is no X and Y, so the (S) value must be calculated (S = ((X * 800) + Y), e.g. to display 3 White pixels starting at X=20, Y=30:
003E9E 0003 ffffff ffffff ffffff

If it's only a single pixel, then the count variable (C) is omitted, and the position variable's (S) first hex digit is set to "8", e.g. to display a single White pixel at X=20, Y=30:
803E9E ffffff

---

Image Messages (for a video background):

• Full-Image message:
  Same as the Image-background, but an extra command is needed to define the visible pixels, V=visible-pixels-info bytes count (4bytes):
  d0 ef 69 00 00 (V)

As for the visible-pixels-info itself, it's the same as the changed-pixels message, but without the PixelColors variable (P), then it needs to end with "00ef69", e.g. to make only 3 pixels visible starting at X=20, Y=30:
003E9E 0003 00ef69

• Changed-pixels message:
  It differ from the Image-background version in two ways:
  1- Just like the Full-Image, the visible pixels needs to be appended to the message
  2- The Alpha channel is somehow added to the pixel colors (BBGGRR) value, but I don't know how exactly, my guess from my tests is that the Alpha channel hex digits replaces the second hex digit of Blue and Green channels.

Note that any pixel not defined in the "visible-pixels-info" will not be shown, regardless of its transparency, and any pixel added at any point can be set to visible/hidden at any changed-pixels message.
In other words, not marking a pixel as visible, does not delete it, just hides it until set to visible again by a changed-pixels message.

---

Notes:

• The app always starts with the "Get_Device" command, might be used to determine the supported commands, or it could also functions as an initialize command.
• There are commands that the app sends but don't seem to do anything (I can skip them with no visible problems).
• All messages must be 250 bytes of size, or multiple of it.
• All commands must be 250 bytes by themselves, so they must be padded without counting other messages within them.
• Every 250th byte is unused, so an extra byte must added.
• Every reply from the monitor (except for two commands, "Get_Device" and "query storage usage") is padded to 1024 bytes.
• The amount of transparent pixels will affect the frame-rate of the video background.
• Some commands can be truncated and still work, but I opted to list them as the app sent them.

[MrFr1day]

Here is a working example (for an image background, and based on how their app communicate with the monitor):

from math import ceil
import serial
import cv2
import atexit

class CMD():
    Get_Device = '01ef6900000001000000c5d3'
    Update_IMG = 'ccef690000'
    Stop_Video = '79ef6900000001'
    Display_Full_IMAGE = 'c8ef69001770'
    Query_Render_Status = 'cfef6900000001'

class Unknown():
    PreImgCMD = '2c'
    Media_Stop = '96ef6900000001'
    PostImgCMD = '86ef6900000001'
    OnExit = '87ef6900000001'
  
def OnExit():
    SendMSG(Unknown.OnExit)
    lcd_serial.close()


def ReadReply():
    response = lcd_serial.read(1024).decode('utf-8')
    print(response)

def SendMSG(MSG, PadValue = '00'):
    if type(MSG) is str: MSG = bytearray.fromhex(MSG)

    MsgSize = len(MSG)
    if not (MsgSize / 250).is_integer(): MSG += bytes.fromhex(PadValue) * ((250 * ceil(MsgSize / 250)) - MsgSize)
    
    lcd_serial.write(MSG)
    return
    
    #I didn't notice any speed difference in splitting the messages, but their app had random splits in their messages ...
    MsgLimit = 111000
    MSG = [MSG[i:i + MsgLimit] for i in range(0, len(MSG), MsgLimit)]    
    for part in MSG: lcd_serial.write(part)


def GenerateFullImage(Path):
    image = cv2.imread(Path, cv2.IMREAD_UNCHANGED)        
    if image.shape[2] < 4: image = cv2.cvtColor(image, cv2.COLOR_BGR2BGRA)

    image = bytearray(image)
    image = b'\x00'.join(image[i:i + 249] for i in range(0, len(image), 249))
    return image

def GenerateUpdateImage(Path, x, y):
    image = cv2.imread(Path, cv2.IMREAD_UNCHANGED)
    width = image.shape[0]
    height = image.shape[1]

    MSG = ''
    for h in range(height):
        MSG += f'{((x + h) * 800) + y:06x}'  + f'{width:04x}'       
        for w in range(width): MSG += f'{image[h][w][0]:02x}' + f'{image[h][w][1]:02x}' + f'{image[h][w][2]:02x}'

    UPD_Size = f'{int((len(MSG) / 2) + 2):04x}' #The +2 is for the "ef69" that will be added later
    
    if len(MSG) > 500: MSG = '00'.join(MSG[i:i + 498] for i in range(0, len(MSG), 498))
    MSG += 'ef69'

    return MSG, UPD_Size


lcd_serial = serial.Serial("COM3", 115200, timeout=1, rtscts=1)
atexit.register(OnExit)

SendMSG(CMD.Get_Device)                     #Skippable 
ReadReply()
SendMSG(CMD.Stop_Video)                     #Skippable if there is no video playing now
SendMSG(Unknown.Media_Stop)                 #Skippable, might be for album playback
ReadReply()                                 #The reply should be "media_stop"
SendMSG(Unknown.PreImgCMD, '2c')            #Skippable, the app pads it using "2c" instead of 00

SendMSG(CMD.Display_Full_IMAGE)
image = GenerateFullImage(r'F:\1.png')
SendMSG(image)
ReadReply()                                 #The reply should be "full_png_sucess"
SendMSG(Unknown.PostImgCMD)                 #Skippable 

SendMSG(CMD.Query_Render_Status) 
ReadReply()                                 #The reply should containts (needReSend:0) to confirm all message are read/deliverd in order

MSG, UPD_Size = GenerateUpdateImage(r'F:\2.png', 30, 50)
SendMSG(CMD.Update_IMG + UPD_Size)
SendMSG(MSG)

SendMSG(CMD.Query_Render_Status) 
ReadReply()                                 #The reply should containts (needReSend:0) to confirm all message are read/deliverd in order

As you have guessed by now, I'm not a programmer by trade, and this is my first Python code, and working with hex, images, and serial-comms in this way, so I want to ask, did I waste my time decoding a known-standard for their changed-pixels messages, or did they actually make their own?

Also, using the code I posted, it takes about 0.6 seconds to convert and send a full image (800x480), is this monitor faster than the 3.5 model?

[gerph]

That's an impressive amount of decoding for the device, and it seems a little different to the interface that we've seen so far. I've got a device on order so fingers crossed I'll be able to play with that 'soon' (for whatever value of 'soon' AliExpress can give me!).

Well done in getting that far - it looks very useful to produce an update to the libraries!

[arthurferrai]

I have one of these, and I think I can help reverse-engineering this display... Which data sniffer are you using?

[MrFr1day]

Thanks for the offer, but as I said in the OP, aside from adding media files to it, I think I've already decoded all the commands, and I've been using it for some time now.

[sambasan]

Thank you very much for the reverse engineering work so far.
I'm currently writing 5" display support for this software. Still at prototype level and can take some time to finish but anyways. The plan is to also support video backgrounds in themes.
I've also got the display and can help reverse engineer the commands a bit further if needed.
I'm using a generic serial analyzer software that can tap into the communication that is happening on USB over the CDC driver.

Findings so far:
All commands requiring path, for example:

• Get a video file size P=Path, the reply is the size in bytes:

6e ef 69 00 00 00 13 00 00 00 (P)

Actually have the path length in characters, meaning that the correct decoding would be:
6e ef 69 00 00 00 (L) 00 00 00 (P)
L: Path length in characters for example for /mnt/SDCARD/video/earth.mp4 this would be 1b (dec: 27)
P: Path itself (ascii encoded to hex)

Additionally I decoded the video upload process and got a simple upload script working. I'll post the full code example a bit later but in general it goes like this:

1. Check if file exists and its size with get video file size
2. Send upload video command (see below), response should be "create_success"
3. Send raw video bytes (in 250 byte chunks / packets, see sendMSG by MrFr1day), response should be "file_rev_done.img_show"
4. Check file size after upload
5. Send init commands again
6. Send play video, response should be "play_video_success"

Upload video command:

6f ef 69 00 00 00 (L) 00 00 00 (P) (V)
L: Path length in characters for example for /mnt/SDCARD/video/earth.mp4 this would be 1b (dec: 27)
P: Path itself (ascii encoded to hex)
V: Video bytes count with endianness flipped so video with 1069828 bytes (105304 hex) would be 045310 hex you can do this with: struct.pack('<i', video_size_bytes).hex()

Code I used to split the video file to chunks:

def read_in_chunks(file_object, chunk_size=249):
    while True:
        data = file_object.read(chunk_size)
        if not data:
            break
        yield data

def upload_video(local_path):
    with open(local_path, "rb") as video_file:
        for packet in read_in_chunks(video_file):
            send_msg(packet)

---

Notes:

I expect these to work pretty similarly for internal / SD card storage with only path being the differing part.

[alexwbaule]

Can i help with this ? I buy it one with 5 pol screen.

I made a usbcap in windows to get commands, etc....

[alexwbaule]

I want to stick with python , im using linux, and i want to make this work on it.

There is some reverse engineering processing or some documentation about the 5 pol device ?

Using the code that @MrFr1day write, change port, its worked. But, there is more commands to get ?

[arthurferrai]

That's what we have for now... The original executable is obfuscated, so it's a mess to read.

[alexwbaule]

Im doing some reverse engineering with the original executable, and wireshark capturing usb packs.

When i finish i will post here.

I found some news about play videos, and another things.

[ErikApption]

@alexwbaule - what is missing from the code from @MrFr1day? I also tried reverse engineering the original code but it has been obfuscated so it was not very helpful imo

[alexwbaule]

Ok ! Lets share what i found until now....

import time
import serial

def ReadReply():
    retries = 0
    while lcd_serial.in_waiting == 0:
        print("waiting to read...")
        retries += 1
        time.sleep(0.1)
        if retries == 50:
            print("I Can't read.. wait 50 times...")
            break
    response = lcd_serial.read_all().decode('utf-8')
    print("Received: [{}]".format(response))

def SendFromFile(m):
    print("Sended: {}".format(m))
    msg = bytearray.fromhex(m)
    print("Size Msg Sended {} and Waiting... {} - {} ({})".format(len(msg), lcd_serial.out_waiting, lcd_serial.in_waiting, lcd_serial.is_open))
    try:
        s = lcd_serial.write(msg)
        lcd_serial.flushOutput()
        print("Size Msg Sended ({}) ({})".format(s,lcd_serial.out_waiting))
        while lcd_serial.out_waiting > 0:
            print("waiting to finish write...")
            time.sleep(0.1)
    except serial.serialutil.SerialException as err:
        print("Error...Timeout {}".format(err))

lcd_serial = serial.Serial("/dev/ttyACM0", 115200, timeout=1, rtscts=1)

f = open('SendReceiveComplete.txt', mode="r")
contents = f.readline()
while contents:
    cmd, line = contents.split("\t")
    if cmd == "0x00":
        SendFromFile(line.rstrip())
        #time.sleep(0.2)
    if cmd == "0x01":
        ReadReply()
    contents = f.readline()
print("Finished...")
f.close()
lcd_serial.close()

With this code (sorry !! im not a python coder or developer) and this file in attachment, you can "play a update image stream".

(stream file)

How i get this file ?

Using wireshark + some commands to export.

First, set this filter on wireshark (and use one port with only one item attached).

(usb.transfer_type == URB_BULK and usb.irp_info.direction == 0 and usb.endpoint_address.direction == 0) or (usb.transfer_type == URB_BULK and usb.irp_info.direction == 1 and usb.endpoint_address.direction == 1)

Second, with the "action" that you want to capture done, you can save this capture. (stop the capture before, and go to File -> Export Specified Packages).

Once you saved the capture, you can run this command to save only the streaming:

"C:\Program Files\Wireshark\tshark.exe" -T fields -e usb.irp_info.direction -e usb.capdata -r All.pcapng

The usb.irp_info.direction inform the direction (0x00 from your host | 0x01 from your screen device), and the usb.capdata is the "stream" itself.

What i found ? just one command that is 7b ef 69 (set the brightness) before the PostImgCMD.

I tried to use the GenerateUpdateImage in a loop, but at the second time that i call, the "needReSend" is allways 1, and i cant update the image (i build 3 equals images, with the number 1, 2 and 3), but using the captured stream, the update works ok, so, there is something missing that i can't say what is LOL....

If someone can help to reverse this stream to some "image file" (maybe doing the reverse of GenerateUpdateImage does)...