Join multiple readtable data with WeirdDuration

I need help in join the readtablefcn with attached files to an exsisting routine so that I can combine multiple table data.

I have file with different format and I join them using below routine. Example

[Adam Danz]

I'd like to understand the problem without having to download zip files, extract them, set up the code to read those files, and then run the code in hopes to understanding the problem. Perhaps you could distill this down to two examples tables and how you'd like to join them (if that's the main issue).

[Matlab]

Perhaps you could distill this down to two examples tables and how you'd like to join them (if that's the main issue).

Yes. That's correct.

I have new files which are added to the result directory but I find it difficult to add to exsisting joining multiple time series i.e. join multiple time series into one.

If you need full code , Please let me know. I can add the Input files and code to analyze the data - I can do that as well.

Thanks

[Adam Danz]

"I find it difficult to add to exsisting joining multiple time series i.e. join multiple time series into one."

So you're reading in the data without a problem but there's a problem with combining the two timetables; is that correct? Could you attach a mat file that contains the two timetables?

[Matlab]

So you're reading in the data without a problem

No. To me format itself is not understood properly.

Could you attach a mat file that contains the two timetables?

Please find the attachment for Input result file + mfile source + files underanalysis + output_data (containts *.mat file)

Data format example

========================================================================

File - dmesg

Time Format : [s.SSSSSS] [ 0.000000]

It's not clear to me how to add running timer counter to output struture

[ 0.000000] tsc: Detected 2100.000 MHz processor

[ 0.000011] tsc: Detected 2112.000 MHz TSC

[ 0.000011] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved

[ 0.000016] e820: remove [mem 0x000a0000-0x000fffff] usable

========================================================================

File - powerd

Time Format : mmdd/hhmmss.SSSSSS [0616/174001]

[0616/174001.507094:INFO:main.cc(316)] vcsid 0.0.2-r3058-2dd3d106bd58a967dca1551808c11f269741d9b0

[0616/174001.507134:INFO:main.cc(322)] System uptime: 4s

[0616/174001.511552:INFO:udev.cc(105)] Watching FD 14 for udev events

========================================================================

File update_engine

Time Format : mmdd/hhmmss.SSSSSS [0616/104022.398157 ]

[0616/104022.398157:INFO:main.cc(181)] A/B Update Engine starting

[0616/104022.398896:INFO:boot_control_chromeos.cc(137)] Booted from slot 0 (slot A) of 2 slots present on disk /dev/sda

[0616/104022.400533:INFO:real_system_state.cc(76)] Booted non-official build.

========================================================================

Please find the attachment...I want data under folder

dmesg

power_manager

update_engine

The task is to analyze timestamp format and combine them under structure joinedtimetable in [joinedtimetable] = get_fieldnames(Content)

[Adam Danz]

After unzipping MathWorks_Help.zip, I see two folders: Input_Resultfilesdata & Src_mfile_Toparse.

Make this super easy for us. Ideally, we should see the problem in two steps.

1. Loading the attached files
2. running a block of code that you provide us.

Instead, I don't know what file to look at, which code to run, and where to look for the problem.

[Matlab]

1. Loading the attached files

This is under matfile1.mat

2.running a block of code that you provide us.

This is under get_fieldnames.m & get_TimeStampCorrection.m files

Note: We have to create function for file"dmesg","powerd.20190616-174001" & "update_engine.20190616-104022" and then execute "get_fieldnames.m" & get_TimeStampCorrection.m

[Adam Danz]

When I run get_fieldnames() with the matfile1 data, I get this error

Unrecognized function or variable 'get_TimeStampCorrection'.
Error in get_fieldnames (line 41)
            [structtable_Date] = get_TimeStampCorrection(Ref_Input,Curr_Input,fieldidx); 

There is a file get_TimeStampCorrection that's not included in the zip.

[Adam Danz]

Matlab 's answer moved here as a comment. Please use the comment section for discussion.

Ahhh! I missed it .Please find the attachment

[Adam Danz]

Ok, I can run the code now.

Now I need to understand where the problem is. Is the problem in this line (line number 55 in get_fieldnames)?

 joinedtimetable = outerjoin(joinedtimetable, structtimetable,'Merge',true);   % 'LeftKeys','RightKeys'

You're merging two table with different column names.

joinedtimetable(1,:)
ans =
  1×3 timetable
    WeirdDuration    log_autoserv1_autoserv    Sublog_autoserv1_autoserv                                      Message_autoserv1_autoserv                                  
    _____________    ______________________    _________________________    ______________________________________________________________________________________________
    00:00:00.000           {'INFO |'}             {'autoserv:0739|'}        {'Results placed in /tmp/test_that_results_hatch_ZByste/results-17-firmware_TPMKernelVersion'}


structtimetable(1,:)
ans =
  1×3 timetable
    WeirdDuration    DBGLog_upstart1_upstart    InnerTime_upstart1_upstart                        Message_upstart1_upstart                     
    _____________    _______________________    __________________________    _________________________________________________________________
    00:00:00.000       {'WARNING kernel:'}          {'[   10.690227]'}        {'init: failsafe-delay main process (681) killed by TERM signal'}

[Matlab]

Any suggestion and help, thank you!!

[Adam Danz]

Hi again, I just spent about 20+ minutes re-reading the comments and going through the code and I still cannot understand exactly what we're trying to fix. I'd like to help but I it's just taking too much time to sift through all of this in order to understand the problem - even at a big picture level.

This is why is important to reproduce the problem with as little code as possible in order to provide a Minimal Working Example. Ideally, you'll get the fastest responses when the problem is described super clearly in just a few sentences and we can copy a short bit of code into our workspace that demonstrates the problem.

Answer 1

[Adam Danz]

In your formatSpec,

formatSpec = '%*6s%13{hhmmss.SSSSSS}D%C%[^\n\r]';

note that you're using lower case hh which is for 12-hour clocks. It works for the update_engine data because the hours are all less than 12:00. But it doesn't work for the powerd data because 17:40:01 is greater than 12:00.

You need to use 24-hour clock notation by using upper case HH.

formatSpec = '%*6s%13{HHmmss.SSSSSS}D%C%[^\n\r]';
%                     ^^

[Matlab]

Super catch. Problem solved .Thanks a lot

Answer 2

[Matlab]

Hi Adam Danz and Guillaume

I have raised a new issue. Can you please look in the below link and please help me . Thank you

https://in.mathworks.com/matlabcentral/answers/499409-error-in-joining-multiple-date-time-duration-id-date-is-in-cell-array