What is missing from MATLAB #2 - the next decade edition

Meta threads have a tendency to grow large. This has happened several times before (the wishlist threads #1 #2 #3 #4 #5, and 'What frustrates you about MATLAB?' #1 and #2).

No wonder that a thread from early 2011 has also kept growing. After just under a decade there are (at time of writing) 119 answers, making the page slow to load and navigate (especially on mobile). So after a friendly nudge; here is a new thread for the things that are missing from Matlab.

Same question: are there things you think should be possible in Matlab, but aren't? What things are possible with software packages similar to Matlab that Matlab would benefit from? (note that you can also submit an enhancement request through support, although I suspect they will be monitoring activity on this thread as well)

What should you post where?

Wishlist threads (#1 #2 #3 #4 #5): bugs and feature requests for Matlab Answers

Frustation threads (#1 #2): frustations about usage and capabilities of Matlab itself

Missing feature threads (#1 #2): features that you whish Matlab would have had

@anyone posting a new thread when the last one gets too large (about 50 answers seems a reasonable limit per thread), please update this list in all last threads. (if you don't have editing privileges, just post a comment asking someone to do the edit)

[madhan ravi]

Thanks Rik!

[Walter Roberson]

Thanks, Rik!

Answer 1

[cui]

Deep Learning toolbox:

How to visualize the changes in dlarray/weight distribution with histograms in deep learning?

Histogram displays how the trend of tensor (weight, bias, gradient, etc.) changes during the training process in the form of histogram. Developers can adjust the model structures accurately by having an in-depth understanding of the effect of each layer.

source

source2

[cui]

Related issues：

visualize the deep neural network architecture ？？？

It seems that Mathworks is not interested in many of the improvement questions (above and below) raised by me, and I regret that there is still no answer!

[dpb]

TMW will not comment in open forum on pending development or toolbox direction; you can be confident they have seen and read the above, but you cannot interpret no response as no interest.

If you wanted more direct input, submit items as enhancement requests, but again, TMW simply does not share its plans in open forums.

[Bruno Luong]

This thread is not an official enhancement request.

To me it's simply a place where people come to emplty the bag of complains.

[Adam Danz]

The first plot is similar to Matlab's waterfall plotting function.

The GIF is too quick for me to make any sense out of it.

Perhaps you could directly suggest these ideas using Contact Us - MATLAB & Simulink

[cui]

@Adam Danz Thank you for your suggestions, I look forward to the official future version will be greatly improved！

Answer 2

[cui]

Deep Learning toolbox:

• action recognition: mmaction2
• video understanding, X3D: https://github.com/facebookresearch/SlowFast ，https://github.com/kenshohara/3D-ResNets-PyTorch
• reID: https://github.com/JDAI-CV/fast-reid
• track: https://github.com/ZQPei/deep_sort_pytorch，https://votchallenge.net/challenges.html
• Metric Learning: https://github.com/KevinMusgrave/pytorch-metric-learning
• image Classification: Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition
• object detection: Centernet, https://github.com/xingyizhou/CenterNet , efficientDet, https://github.com/google/automl/tree/master/efficientdet，detectron2，https://github.com/facebookresearch/detectron2 ,PointRCNN,https://github.com/sshaoshuai/PointRCNN
• face recognition: insightface, https://github.com/deepinsight/insightface, sphereface,https://github.com/wy1iu/sphereface, arcface,https://github.com/ronghuaiyang/arcface-pytorch
• Semantic segmentation, instance segmentation，https://github.com/facebookresearch/maskrcnn-benchmark , Semantic Segmentation (Most of them are difficult to implement with matlab)
• deeplearning preprocess data: https://github.com/NVIDIA/DALI
• StyleGAN: https://github.com/NVlabs/stylegan
• DeepNetwork Architecture：Mnasnet，https://github.com/tensorflow/tpu/tree/master/models/official/mnasnet , Bayesian-Neural-Networks , Graph Convolutional Networks
• similarity search: https://github.com/facebookresearch/faiss
• deepfakes: https://github.com/deepfakes/faceswap
• deepnetwork inference performance: https://github.com/microsoft/onnxruntime ,https://github.com/zuoqing1988/ZQCNN
• image Labeler app: https://www.mathworks.com/matlabcentral/answers/789159-how-does-image-labeler-app-realize-this-interactive-operation-by-writing-a-program?s_tid=srchtitle

and so on....

The above are influential applications of deep learning in various aspects, but it is difficult to reproduce in matlab. Although Matlab2019b version supports automatic differentiation mechanism, it is still difficult to implement algorithms in matlab. The efficiency of the differentiation mechanism is not high, and many operators do not support it. I tried to implement the more famous yolov3/v4 algorithm with the latest MATLAB2020a version, but it is still not satisfactory

In summary, my personal suggestions are like my personal answer above, and I hope that future versions can improve a lot!

以上都为深度学习在各个方面有影响力的应用，但是在matlab中复现困难，虽然Matlab2019b版本支持自动微分机制，但仍然不易在matlab实现算法，微分机制效率不高，很多operators也不支持。。。

总之，我的个人建议就像上面的个人回答建议一样，我希望将来的版本可以有所改善！

Answer 3

[Rafael S.T. Vieira]

I would love to have a command TeX2sym and sym2TeX. With it, we could bring formulas from LaTeX and run them in MATLAB and the other way around. It is a tedious work doing it by hand, and I could bet most MATLAB users do it or will do it eventually.

Another useful feature to add would be arbitrary-precision arithmetic...languages such as python and Java (with Bignum), allows unlimited precision when dealing with integers at least. Granted that It is slow, but I believe that the MATLAB team could do something better. And I honestly feel like MATLAB is missing a feature by not having it...even some competing software has it.

And finally I was considering buying the Text Analytics Toolbox, and it would be nice, if it could have a grammar/spell checker. Even if it is not as advanced or doesn't contain all words. With live scripts, we can write an interactive document, so it would be nice if MATLAB could correct our spelling (even if it required a ToolBox to do it).

[Rik]

Neat idea. With enough patience it shouldn't be too hard to write a basic tex2sym, but I don't have any good ideas for sym2tex that wouldn't require a lot of cleaning up.

I think the main problem with tex2sym would be interpreting the subscript and superscript. The main problem with sym2tex would probably be that a clear and concise mathematical description is not necessarily the same as compact Matlab code, even when working with symbolic variables.

[Walter Roberson]

sym2tex exists as latex()

tex2sym is pretty hard. Pretty mathematical notation is often ambiguous.

[Walter Roberson]

For example

• sin x+y -> %observed in the wild. People are likely to interpret as sin(x)+y . Not recommended latex style for trig functions
• \sin x+y -> %is it sin(x+y) or sin(x)+y? People are likely to interpret as sin(x)+y
• \sin{x+y} -> %sin(x+y) but people are likely to guess sin(x)+y without being sure
• \sin{x}+y -> %sin(x)+y but people are likely to guess sin(x)+y without being sure
• sin(x+y) -> %sin(x+y). Not recommended latex style for trig functions
• \sin(x+y) -> %sin(x+y). Better latex syntax for trig functions
• \sin\left( x+y \right) -> %sin(x+y) . Considered better latex style because it allows latex to match heights of () when internal content is varying sizes, but has too much space
• mx+b -> %m*x+b which people understand from long convention. But it could be a variable named mx that is being added to b . This is, however, the primary recommended latex style
• m.x+b -> %m*x+b but not recommended latex style. However, lower dot for multiplication tends to show up in papers that have matrix multiplication
• m*x+b -> %m*x+b but actively recommended against latex style
• {m}{x}+b -> %m*x+b which people understand from long convention, but visually it could be a variable named mx that is being added to b. Easier for mechanical interpretation but not typical latex style
• m{\cdot}x+b -> %m*x+b understandable both mechanically and human. One of the recommended latex styles
• m{\times}x+b -> -> %m*x+b understandable both mechanically and human, but less common in practice. Considered to be one of the valid latex styles if needed
• f{\textrm{\"{o}}}rl{\textrm{\aa}}t -> %a single word. Using {} inside an expression does not reliably indicate multiplication. And it was harder to get the special characters to work here than it should have been

Implied multiplication without any syntactic separation is very common in mathematics and latex.

The above shows just some of the ways that simple operations can be written in Latex, and tex2sym would have to try to understand them all.

There is also the issue that a lot of latex uses constructs that MATLAB does not support, especially \usepackage and amsmath mode.

[madhan ravi]

Wow , deep sir Walter!

[Rafael S.T. Vieira]

Thank you for your interest, Walter and Rik. I believe that It is best to teach ourselves to code in a particular way, which will allow MATLAB to convert it to sym, than to be doing the task by hand every time.

Operator precedence could take care some of these issues. If \sin x + y was the input, then left-to-right precedence could dictate the output to be sin(x) + y. And to obtain sin(x+y), we would have to code in LaTeX \sin{x+y} or \sin(x+y).

Implied multiplication is indeed very common in mathematics and latex. On the other hand, it is also almost ubiquitous that variables are written just as single letters (especially if we are writing implied multiplication).

Finally, a command like tex2sym does not need to contemplate every math package imho, just some set of commands and macros. MATLAB could even return its best guess, and let us do the remainder of the task. Of course, ideally, we would just copy the contents from environments like $$ and \[ ... \] and paste them into MATLAB for a tex2sym conversion.

Answer 4

[dpb]

Editor won't restrict a substituion to selected area...best it knows of is the function. Uncool in the max! I'm now having to fix up a bunch of stuff shouldn't have to have done... :(

Answer 5

[Shae Morgan]

facet_wrap or facet_grid (or general ggplot2 functionality) version of subplots, or some altered, simpler customizability for subplotting subsets of data.

gramm is an available toolbox, but it'd be nice to have it built in

Answer 6

[Seth Wagenman]

Ability to convert Python items to MATLAB data types (other than numpy arrays) inside of MATLAB, rather than inside of Python using the admin-rights-required API: https://www.mathworks.com/help/matlab/matlab_external/install-the-matlab-engine-for-python.html?s_tid=srchtitle

Answer 7

[Walter Roberson]

A "soft interrupt" -- that is, an interrupt that can be handled with try/catch .

For example I am running a long calculation at the moment that iterates several times, and each iteration can take more than an hour. I don't always have the patience to wait through as many iterations as I asked for. For my purposes, the output of the last successful iteration would be "good enough" if I were to ask to interrupt the function.

I can control-C, but that interrupts the function completely, losing all of the outputs, and losing the function variables.

If I could somehow "soft interrupt" and have it return the current values of all variables, then that would be good enough for my current task. But the generalization of that would be the ability to catch a soft interrupt in a try/catch so that the code could make whatever final summariziation it needed to in order to create usable output variables. Furthermore, even though my own routine might be happy to return the "last good" values of the output variables, any soft-interrupt I requested might well get received while some lower-level routine had control that did not know about soft-interrupts, so I would want interrupt (with no return values) to apply to those layers until control reached my handler for a clean run-down.

[Steven Lord]

This isn't exactly what you're asking for but you could pause your running program (which may pause in a lower-level routine.) Then select your function's workspace and save the variables you want to keep. Or if you want MATLAB to continue the current iteration and stop just before it starts the next, add a breakpoint on the line where the next iteration will start and continue the current iteration. This functionality has been part of MATLAB since release R2016a.

[Walter Roberson]

It can be a pain to dig into the program each time, and requires detailed knowledge of the program.

... I've already held off several releasing this program because I am not happy with the command line interface I have for it (too many obscure options that are rarely (but sometimes) useful)... I don't expect people to manually poke through the code to put in breakpoints and know which variables to save... but it would be common for people to get tired of the program executing for hours and to hope not to lose the progress so-far if they interrupted. Yeah, I could save to global variables, but...

ICK

[Rik]

Another solution in current Matlab would be a GUI with a flag controlled by a button. But I believe you have previously talked about a function that would need a manual for a GUI to be feasible.

That last section sounds exactly like try-catch, so essentially you want to be able to trigger an error, which will then cascade up to your softtry-catch block. Either way this sounds like a useful feature.

[Walter Roberson]

a function that would need a manual for a GUI to be feasible

That's probably the same program being referred to ;-)

[Bruno Luong]

Just program an onCleanup to save whatever that needs to be saved. Ctrl-C do the rest.

What you propose is a dream but how to deal with variables/outputs not assigned and function interrupted at unknown state?

Only the programmer can know his workflow. So no possible to implement this lazy feature IMO.

[Walter Roberson]

onCleanup would have to either save() or store into global variables. The point is to be able to have a smooth return of values.

Outputs not assigned would still generate the same error messages as at present.

Function interrupted at unknown state is no worse of a problem than functions that error and are caught by try/catch at present.

It is common for programming systems to have a control-C handler .

[Sindar]

My solution in a similar case was to check whether a certain file ('FAIL_MODE.txt') exists in the directory at the end of each iteration. If it does, break the loop. Then, to stop the program, all I had to do was create the file (which is pretty trivial unless Matlab has frozen the whole computer)

Answer 8

[Tim]

The following changes to the internal volume renderer would make Matlab much more useful for volume visualization:

• True RGB volume visualization (vs. just scalar data + colormap)
• Independent, decoupled specification of alpha values and intensity values
• The ability to combine volume images with standard axes and axes objects like points, lines, patches, etc.

Answer 9

[Mario Malic]

When typing code, if user wants to reference a variable or a function, one could click tab and get a matching list of functions and variables. Would it be useful to split this functionality with modifier if user wants to reference one of the two? As an example Shift +Tab for variables and Tab for functions and variables (not to break the current functionality).

Answer 10

[cui]

MATLAB

I very much hope that the official version will strengthen the readstruct function in the future! Lack of complete uniformity to support more format requirements.

How to effectively read ".yml" files as structures(and similar '.json','.xml',and so on)?

Answer 11

[Camilo Andres Manrique Escobar]

One thing would be: defining classes in notebooks (Live Scripts in MATLAB case). That's a normal thing for years now in Python, for example.

Answer 12

[Mikhail]

Some (well, most) dynamic programming languages allow for an experienced developer to have an insight look into how their code is actually executed by the runtime environment.

They are able to see the parse tree and the compiled byte code.

They are able to see what parts of code are JIT-compiled, and how exactly the are compiled. When something is not compiled, they are able to see why.

The developer doesn't have to guess whether a particular optimization has kicked in or not. They know for sure how each and every object in their code is handled (whether it is CoW, passed by reference, passed by value).

I'd love to see these capabilities in MATLAB, too.

[Rik]

The problem with this idea is the long term portability of code. It is fairly easy to write code that will run on Matlab 6.5 (released in 2002) and R2020a, as you can see in my FEX submissions (although some of those functions are examples of when it is hard to so). If you want to optimize for the JIT engine of every release, you blow up your dev time. If your code runs 10% faster because you take more advantage of the JIT, but you spend weeks on that function, is it worth it?

The engineers at Mathworks are very aware of good coding practices taught to Matlab users, and tend to try to optimize for such code. There are absolutely more obscure parts of Matlab that allow speedups in unexpected places: cellfun(@isempty,data) versus cellfun('isempty',data) is an example of that.

As a last aside: diving deep into the inner workings of Matlab might be blocked by Mathworks for a completely different reason: if you can see how everything is put together it is possibly easier to reverse engineer, allowing competitors to leverage engineering time from Mathworks in their own products. I have no clue whether or not this is a consideration (since I don't have any insider knowledge, nor personally know anyone working at Mathworks), but it might be.

[Mikhail]

I'm not talking about optimizing for JIT. Having insight into what exactly JIT did, a developer can understand the reason why their code e.g. became slower in a newer release. Or why a particular construct is faster than another (without speculating with other MATLAB users as to why this is happening - which I seen here at FEX quite often).

You also seem to have missed my point about parse tree. This information can be usefull and can spawn a wide range of third-party MATLAB utilities - such as code analyzers, linters, profilers, dependency analyzers and so on.

But you are right that this is probably what MathWorks tries to avoid.

But having said that, there should be no doubt that any professional programmer will choose a decent programming language (such as Julia, R, Python or even Lua), which doesn't limit his (programmer's) knowledge and urge to dig as deep as he wants to dig.

If MathWorks wants to attract those professional programmers and not only position MATLAB as a "language for engineers and scientists", they will have to open up.

[Bruno Luong]

"If MathWorks wants to attract those professional programmers and not only position MATLAB as a "language for engineers and scientists", they will have to open up."

Unfortunately I think this won't happen. I have observed them to close down (and not open up) more and more over the years. Examples come to mind is: The mxArray internal structure was hidden, they removed access to undocumented functions, they advises user to code normally and should not care about understanding why some method is faster or slower, telling speed is their business.

Yair's undocument webpage has a huge success not without reason. And I can imagine it take a lot of effort to make it up-to-date.

What you request, and I'm 100% second, seems to hane negative correlation coefficient with TMW past/ current politics. They might have a reason to do it, mainly prevent plagiagism from competitors.

[Walter Roberson]

When trying to understand these kinds of decisions, I find that it helps to keep in mind a couple of factors:

1) The purpose of the Free Software Foundation projects such as Octave is not to provide high quality or long term supported or "affordable even in developing countries" software. The purpose of the FSF is explicitly political: to make it effectively impossible for anyone anywhere in the world to "own" (control) what is commonly known as "intellectual property". The FSF is explicitly intended to destroy the economy of companies such as Microsoft, and Apple, and Mathworks that is based on "owning" the rights to sell software licenses. The FSF believes that once an intellectual idea is produced and instantiated, that it belongs to the entire world and should be available to everyone (**with** source code) for no more than a service fee for the time/resources to copy it. To the FSF, it is not necessary that companies such as Microsoft and Mathworks cease to exist, but rather that such companies should earn their income by providing consulting services to people who want modified versions of the software, or want programs written (with the new software being released in source code form to the public) -- or through sidelines in mechandising physical objects such as Mathworks-branded t-shirts, or Mathworks-branded toys for McDonalds Happy Meals. Therefore Mathworks is not just competing in a marketplace of providing high quality software that users are willing to pay for: Mathworks is competing in a marketplace where one of the significant players is determined to force Mathworks to give away the many millions of dollars invested into intellectual property development, for political reasons. The FSF actively wants proprietary software firms to fail in anything recognizable as their current forms, and the FSF has no plans to mitigate problems that would cause in the world: the FSF just has the nebulous idea that if software is worth doing, then someone will be willing to hire programmers to develop the software (that would then have to be given away to everyone.) Thus, to companies such as Mathworks, the FSF is not just a "respected competitor", it is a malignant actor determined to destroy the right of Mathworks to exist at all (except as a consulting firm.)

2) Google vs Oracle America https://en.wikipedia.org/wiki/Google_v._Oracle_America is an active legal case in the USA. To summarize very briefly: Sun developed Java and encouraged its wide adoption as a portable write-once platform; companies did that, including copying the major java interface components but writing their own version of the internals; Oracle bought Sun; Oracle now claims ownership of the very interface structure for Java and is trying to get companies to pay big $$$ for licensing what Sun once encouraged everyone to use. Oracle sued Google for $US8.8 billion in copyright damages with respect to Android. Google won in lower court, but the decisions were reversed in a higher court, and the matter is now on the docket of the US Supreme Court.

The relevance of this lawsuit to Mathworks is the question of whether Mathworks "owns" the structure and library interfaces it established for MATLAB (and so could potentially force FSF Octave project to modify or shut down). Likewise, if Mathworks were to expose the Execution Engine level, would Mathworks effectively be giving away those decades of intellecutal work for anyone to develop for free, or would they retain enough control to be able to prevent their competitors and organizations such as FSF from using the work against Mathworks? I know that if I were in the position of a company such as Mathworks, that I would not be willing to release significant information about internal infrastructure until the question of ownership / control of APIs was legally settled.

[Rik]

(sorry for the wall of text)

I don't understand why people resort to calling the Matlab language bad (even if only by inference) if they disagree with the direction Mathworks has chosen. I don't think R is better as a language, or Python. They have their pros and cons. If I think a language has a con, why should I start calling it names? Do I have a right to say Python is stupid because you need to specify what exact version of what exact package you want to use, because otherwise everything will break or at least run the risk of not being able to reproduce the result for a paper? Of course not.

I don't think Matlab (as a language) is not mature. You may not like the direction that Mathworks is choosing to move the language, but that is something else.

It makes sense to hide implementation details on some level. Every time I post a function to the FEX I do exactly that. I tell people to use readfile(filename_or_url) and use that result, instead of telling them how to figure out if a file is UTF8 or ANSI and which exact conversion that requires for their specific OS and version of Matlab/Octave. If you want to know, read the m-file. I could have chosen to publish the function as a p-file, but I didn't. That was my call to make. The function is not suddenly horrible because you can't have a peek at the implementation and optimize your workflow accordingly. Why would the language as a whole be any different?

More on topic: I don't see why you would want to understand what the JIT is doing to your code, without wanting to optimize for that. What would be the use? I'm all for creating more understanding, but I don't see how this achieves more than what good coding practices would teach you.

I have no experience working with a parse tree, so I can't comment on that.

About that 'any professional programmer': that reeks of a true Schotsman fallacy. Apart from what I mentioned above, I don't see what more I could say. It sounds more like venting frustration than actually making a point.

@Bruno, it makes sense on some level to let speed be their business. It litterally is. GNU Octave has fewer debug tools, fewer implemented function, but most importantly for most people: it is a lot slower. If you write code following the guidelines everyone is being taught on this forum and in the documentation, Mathworks can focus engineering effort on only a single style of programming. conv used to be single-threaded, now I can saturate 12 threads with a convolution. The ismember function used to create memory issues for large amounts of text in a cell array (my use case: about 3-4 MB), now it works fast. The more you open up your internal mechanisms, the harder it is to make the fundamental changes required for orders of magnitude speedups.

None of this is to say that I am a big fan of every decision Mathworks makes. I just think there is enough valid criticism to level that you don't need to vent frustration under the guise of commentary. Just be honest with yourself and others.

[Walter Roberson]

The parse tree can be accessed through the undocumented mtree() function https://www.mathworks.com/matlabcentral/answers/180048-list-built-in-commands-used-by-m-function#answer_169083

[Mikhail]

> I tell people to use readfile(filename_or_url) and use that result, instead of telling them how to figure out if a file is UTF8 or ANSI and which exact conversion that requires for their specific OS and version of Matlab/Octave.

> ...

> The function is not suddenly horrible because you can't have a peek at the implementation and optimize your workflow accordingly. Why would the language as a whole be any different?

Given that I can't peek at the function's implementation, this function suddenly becomes horrible when I have a file double encoded to UTF-8 (and so your function returns garbage) and you considered your users too dumb to allow them to mend this function for their needs.

Same goes for the language.

[Rik]

That doesn't make the function horrible, it just means the function is not suited to your needs. I don't complain that my hypothetical electric stove can't cool things down, even though with a TEC it is possible to both heat and cool down things with a single electrically powered device.

Although you could re-use a lot of the internals of readfile to fix double UTF-8 encoded files, the problem is not the function, it's the double encoding. Fixing double encoding is a different task from reading a file correctly. It wouldn't be a matter of 'mending my function', it would be re-using internal functions for a different goal. The word 'mending' implies my function is broken, but returning a double encoded file in a corrected form is not living up to the contract that the name of the function is offering.

Also, there are more reasons to close down the source than just thinking your users are dumb (suggesting otherwise sounds a lot like assuming bad faith). As an example: if you want to provide a function for a license fee, it would be plain stupid to have your license check in an m-file, since it would be almost trivial to circumvent.

Answer 13

[dpb]

A ready-built insertion method for tables, arrays, etc., so don't have to do all the under-the-hood indexing manually...given an array of the right size and matching index, if had table (say), then

tMyTable=insert(tMyTable,indices,tNewData);

ideally, an option with keyword 'Empty', T|F would be available as well.

Maybe I'm missing some magic pixie dust, but I've been unable to figure out a way to this without physically moving a section at a time to insert the new row which means working from rear to front to avoid changing the indices to the insertion points or by catenating the pieces starting from the front.

[Walter Roberson]

Hmmm... what would the Empty option do?

[Walter Roberson]

You can do it without moving a section at a time by building a cumulative destination index that increases by the size of each gap at the appropriate places (though I need to think more about how to vectorize this step.) Then you assign the existing values into a new variable, destination indexed by the cumulative index. Then you can assign the new values into the gaps with a single assignment statement using a destination that is just the "holes".

Not sure exactly how to vectorize the index construction... but the data moving itself does not have to be done a section at a time.

[dpb]

"what would the Empty option do?"

Insert blank record(s). With table you have to construct a record of the right size and type for each column and with the matching variable name; would just be "syntactic sugar" to have the routine automagically handle that as well. (Probably, the keyword is enough, likely the only use would be with the value 'T')

Useful if you know where the new record(s) goes(go) but don't yet have the data for it(them) at hand but will be delivered on the next train to arrive at the station. Or, if only have part of the full record with the rest not arriving until the container ship from China gets to port. Insert the blanks then fill in the fields later.

Possibly there's an 'Incomplete' option that would be the composite of the two steps for the partial data in hand case?

Not that big a deal for an array of native type, agreed, but have run into the case enough times with tables for the thought should be a builtin -- at least for the table/timetable/timeseries(?) classes.

[dpb]

Yeah, that's undoubtedly a better way on the construction, Walter -- I hadn't really thought about the implementation too much other than going "dead ahead" to "just get 'er done!" for the immediate task at hand.

Taking a break, I thought I'd throw it out as an idea for an enhancement for discussion...with the sidelight maybe somebody did have a "trick" hadn't thought of.

Answer 14

[Walter Roberson]

I know I've said it before, but it's still missing and still important:

We need a way to gather all of the outputs of function call into a cell in the middle of an expression .

I know this may be tricky to implement internally. There are internal rules that are hard to work out, that have to do with how many outputs to request from functions. For example,

[A, B] = cellfun(@C, D, 'uniform', 0)

somehow passes knowledge of "two outputs" to C -- for example if you were to use @max then A would be a cell array of the maxima and B would be a cell array of the indices. The situation can get more complicated than this, and figuring out all the cases can make your head hurt. But we do know that any expression C(D(E)) that D(E) will be evaluated asking for exactly one output and that would be passed into C... but the knowledge of multiple outputs would be passed to C and yet not D.

The number of outputs to use is not inherently clear. If for example you call ode45 and ask to gather the outputs, are you asking for the common TOUT, YOUT case, or the full TOUT,YOUT,TE,YE,IE ? There are some cases where extra outputs can be expensive to calculate, so even though an operation that gathered "maximum" outputs might be useful, it is not always desireable, so ability to select the number would be useful.

Then there are issues with, for example, deal(), where you can have any number of outputs with just a single input:

[A,B,C,D] = deal(123)

would initialize A, B, C, and D all to 123. So if you ask to gather "all" of the outputs from deal(123), that number is not well defined.

Working these things out is not trivial -- but it is a really missing bit of the language.

There might be an opportunity for a syntax such as {}name -- e.g.,

arrayfun(@(X0) {}ode45(@f, tspan, X0), x0s)

meaning to gather all of the outputs of the call. At present, {} is not valid before a name.

[Bruno Luong]

Why nargout doesn't meet what you ask?

function varargout = foo(varargin)
varargout = cell(1,nargout);
[varargout{:}] = deal('dummy');
fprintf('request with %d outputs\n', nargout);

Test

>> [A]=cellfun(@foo, {1 2 3}, 'UniformOutput', false);
request with 1 outputs
request with 1 outputs
request with 1 outputs
>> [A,B]=cellfun(@foo, {1 2 3}, 'UniformOutput', false);
request with 2 outputs
request with 2 outputs
request with 2 outputs
>> 

[Walter Roberson]

Bruno,

Imagine that you want to implement

[temp{1:3}] = ndgrid(-1:.1:1);
C = cell2mat(cellfun(@(M) M(:), temp, 'uniform', 0);

That is, you want a single array in which each of the columns is one of the outputs of ndgrid(). And you want to do it as a single expression.

ndgrid() does not have a fixed number of outputs -- it is not like sin() with one fixed outputs, or max with two fixed outputs. nargout(@ndgrid) is -1 -- in other words the declaration is like

function varargout = ndgrid(varargin)

If you wanted to capture all of the outputs of max() then you could query ndgrid(@max) to get 2:

function outputs = gather_outputs(f, varargin)
  n = nargout(f);
  [outputs{1:n}] = f(varargin{:});
end

and you could

 gather_outputs(@max, rand(3,5))

and this would be fine for gathering the two outputs of max into a single cell array.

But if we try to

 gather_outputs(@deal, [])

then nargout(@deal) is -1 and if we said "Okay, take the absolute value of that -1" then we would be doing

 [outputs{1:1}] = deal([])

which would give you just {[]} as the output.

This shows that you cannot just look at nargout() of the function you are invoking, such as @max or @deal .

Can we just look at nargout of the overall expression to determine the number of outputs to use for deal? No,

 C = cell2mat(cellfun(@(M) M(:), gather_outputs(@ndgrid, -1:.1:1), 'uniform', 0));

would at best tell you that nargout is 1 (the C variable). You need something else to tell you the number of outputs you want to get -- something like

function outputs = gather_n_outputs(f, n, varargin)
  [outputs{1:n}] = f(varargin{:});
end

and then you could

 C = cell2mat(cellfun(@(M) M(:), gather_n_outputs(@ndgrid, 3, -1:.1:1), 'uniform', 0));
 

The request is to build this kind of facility in to MATLAB instead of having to write true functions like gather_n_outputs and have to pass function handles into them. Some syntax like

 C = cell2mat(cellfun(@(M) M(:), {3}ndgrid(-1:.1:1), 'uniform',0));
 

where the hypothethical new syntax {3} indicates that you are requesting 3 outputs and that you want them gathered in a cell array. A common alternate syntax would be {} to request all of the outputs, like

 C = cellfun(@(V) V.^2, {}max(rand(3,5)), 'uniform', 0)

which would hypothetically gather both (all) outputs of the max() call into a cell array that would then be available for further processing.

If you just used

  C = cellfun(@(V) V.^2, max(rand(3,5)), 'uniform', 0)

then the knowledge of the single output would be carried through into the cellfun, which would tell max() to only emit a single output, so you would not get the second output processed. And the first output would be numeric not cell, so you would need arrayfun instead of cellfun...)

Answer 15

[Walter Roberson]

A "select" clause for readtable() and kin.

For example one user only wanted to read the rows in which one particular variable had a particular value

https://www.mathworks.com/matlabcentral/answers/610491-how-to-select-specific-content-in-text-file-and-read-it-into-a-matrix

It could at some point be implemented in terms of a rowfun() type function that got passed the variables for a row and could make arbitrary decisions based upon the row contents.

However an earlier stage could potentially have pairs of variable names (or numbers) and a vector or cell array of values, in which the selection code did an ismember(). Such a facility could be further improved if there were a "sorted" option (so the code could figure out when to give up looking -- any one value should no longer be looked for if a larger value were encountered). Or even "grouped", which would not imply sorted as such but would imply that when you find a value that all the instances of that value that exist will be in adjacent rows and so as soon as you detect a change you can know to give up looking for that value.

Answer 16

[Sindar]

Ok, this is pretty minor, but:

For common functions that return values as the first output and indices as the second, it would be nice if there was a direct way of getting the indices, so I could do things like this:

x = [1 2 5 4 5];
y = 1:5;
y_xsorted = y(sortInd(x));
% or
y_xunique = y(unique(x,'ia'));

instead of needing to create temporary variables:

x = [1 2 5 4 3];
y = 1:5;
[~,idx] = sort(x);
y_xsorted = y(idx);
[~,idx] = unique(x);
y_xunique = y(idx);

I know I could make wrappers myself, but this seems like a case where a builtin function could potentially be noticeably optimized. (And, is more straightforward than rebuilding the whole output system)

Answer 17

[Bruno Luong]

Implement family of (numerical) data structure with O(1) inserttion, removal, that includes chain-list, binary tree, Fibonachi tree, red-black tree etc, etc, ... the performance must be focus point. Don't care if they are encapsulated or not in the OP, just don't make that reduce the performace.

Answer 18

[Bruno Luong]

Implement equivalent to C inline-function (or macro) so as calling this function on small data won't be penalized in speed with the over-head.

Answer 19

[Bruno Luong]

No big deal but I wish SIGN(X) function could return 1 for X=0 instead of 0. May be implementing an option to not break the compatibility.

I just rarely use SIGN because of this exception choice.

[Adam Danz]

In such cases I've used, sign(x+realmin)

Example,

sign((-2:2)+realmin)

ans = 1×5

-1 -1 1 1 1

●

[Walter Roberson]

Wanting sign to return 1 at 0 would seem to be for use as Heaviside function.

Heaviside is available in the Symbolic Toolbox as heaviside() -- though you would want to sympref('heavisideatorigin', 1).

For numeric work it can often be replaced by (x>=0)... with the usual warnings that if what is being multiplied is nan or inf then you get nan out... but that would happen for sign() as well.

[Adam Danz]

sign() returns the sign of -inf/+inf (-1/1) and returns NaN for NaN inputs.

[Walter Roberson]

It is common for people to write terms in the form

something + value * u[x-offset]              %unit step function

and common for people to implement that either as

something + value * heaviside(x-offset)     %OR
something + value .* (x>= offset)

and occasionally as

something + value .* (sign(x-offset)+1)/2

except wanting sign() to return 1 at 0 to achieve the same boundary condition.

And that is fine provided that value is finite when it is unselected, with non-finite being a problem because 0*inf -> nan, 0*-inf -> nan, 0*nan -> nan.

This is not a problem that can be cured by using the heaviside() or sign() functions or the >= operators... it needs something more like piecewise:

something + piecewise(x >= offset, value, 0)

It is not a problem in theoretical mathematics because Heaviside is more a distribution than a function (just like Dirac Delta is a distribution) and Limit of the distribution acts to remove the problem.

[Bruno Luong]

Why I prefer sign(0) = 1?

To me y = sign(x) should satisfy these two important properties

norm(y) = 1
x = y * abs(x)

This uniquely determines y for any x ~= 0 (real, complex or even quaternion). For x=0, any y=exp(1i*alpha) with alpha real meet both properties, so just select it as y=1 for arbitrary convention.

However the MATLAB SIGN() function does NOT satisfy the first property for x=0.

Adams's trick sign(x+realmin) is interesting, but it just shift the issue to x=-realmin.

Answer 20

[cui]

Deep Learning toolbox:

groupedConvolution2dLayer since R2019a support, but groupedConvolution3dLayer don't support ??? (until R2020b)

[Rik]

Maybe you could merge this answer with your previous answer with deep learning toolbox suggestions? I would also encourage you to contact support with these suggestions. It seems you have a lot of them, so they might be interested in hearing from you.

[cui]

Mathworks-China does not actively care about my enhancement needs, but is interested in my work unit and personal contact number. I am very disappointed!

[Adam Danz]

@cui could you elaborate on your last comment? How did you contact MathWorks and why do you have that impression of them?

Answer 21

[Robert Guldi]

As far as I can tell, there does not seem to be very much in the ways of documentation for Excel-based work (i.e creating and manipulating charts). I'd love to see some of this material added into the Documentation search if at all possible.

[Adam Danz]

Are you referring to tables or timetables, readtable, readcell, readmatrix, readtimetable, textscan | writetable or perhaps some of the support functions such as readvars, setvaropts,detectImportOptions, Import Tool, preview? Also see

• Access Data in Tables
• Import Spreadsheets

[dpb]

Robert specifically mentioned creating and manipulating charts -- and he's right, there are no builtin facilities for handling more than reading/writing data to/from Excel.

I don't know/think that it is TMW's place to be writing MS Excel documentation translation to/from Excel, but there are a lot of people apparently trying to manipulate objects such as charts inside Excel documents from/with MATLAB.

I've just been through the process of simply trying to format an automated template created from MATLAB code and again, while it's almost trivial to build the data and formulas and write them with writecell or writetable (and, if one looks, one discovers all writecell does is call cell2table and then pass the result to writetable), but then one has to reopen the file with an Excel COM object and mung in the object model to do anything else like set number formatting.

Or, I discovered that if one needs to construct the spreadsheet piecemeal (say doing an agglomeration of data across a large collection by the unique IDs present so one gets those at one time instead of all at once) that even with writetable, the overhead of opening/closing the file is, at best, painfully slow for more than just one or two sections, or at worst, completely hangs the system.

To accomplish this goal, one has to revert to a hack of opening a COM object and writing all the data before then saving and closing the file. At least with xlswrite one has the outline of a code to do that that some others have already extended on FileExchange to do that.

Doing so reduced a process from over an hour with writetable when it did actually finally(!) run to only 7-10 seconds.

While I'd be happy if I never had seen Excel for the most part, there are millions using it and I'm stuck as a volunteer for local nonprofit with trying to make some of their manual processes built around a plethora of spreadsheets at least somewhat more productive by packaging applications to process these data for them.

Fortunately, I haven't had to deal with Excel charts (at least so far, althoug that might be on the horizon), just the data that is much simpler to pull from the dozens of workbooks, assimilate with MATLAB tables and vector operations with rowfun and friends and then put back as compared to trying to write the same functionality in VBA. But, the office folks still have to use Excel so can't just throw it away -- all a long-winded way to say I can relate to the wish and understand how it might be a need, not just a preference instead of plotting in MATLAB (and, truthfully, for some kinds of management data, the abilities packaged in Excel are light years ahead of TMW/MATLAB in functionality and appearance).

[Adam Danz]

I wasn't aware that Excel charts can be mainpulated with Matlab or that anyone would even want to do this but I realize there are lots of different workflows outside of my own. I don't understand how it would be advantagous to use Matlab, a much more powerful tool than Excel, to create charts in Excel. The skill-set needed to do that seems greater than the skill-set needed to create a plot directly in Matlab. But you bring up a good point about programmers forced to make things work for people using Excel.

Manipulating sections of a spreadsheet or workbook is understandable and since Matlab does have functions that can read/write to specific sections of worksheet, I didn't realize there was anything more that was needed.

My first programming was done in the mid 90s with TI Calculators using a program that could upload code written in an IDE to the calculator. Then I started programming macros in Excel for years. But as soon as I learned Matlab, the rest was history concerning Excel/TI.

Matlab documentation on Python integration is growing. I wonder if there's a team that focuses on Excel integration.

[dpb]

"My first programming was done in the mid 90s with TI Calculators using..."

Don't get me going on geezer stories or I'll be reminiscing about Wang nixie tubes and even further back... :)

https://www.oldcalculatormuseum.com/wang360.html

[Adam Danz]

Now I want one.

[Robert Guldi]

I'm working specifically with ActX server, and I eventually figured out everything I needed to get my program to work. While it is certianly easy to plot things in Matlab, I needed the data (which started out as a TDMS file) to be easily manipulated and viewed by people who have are very comfortable with Excel and want to be able to view and edit the charts and data with ease, much like dpb was talking about. MATLAB is very powerful and it can process the chart changes with hundreds of thousands of datapoints each much faster than Excel can, as it has to regenerate the chats any time there is a change, and it's reduce the total time to process by a factor of 10, and I think any better documentation would greatly help a very large number of people.

[Mario Malic]

I wonder if there's a team that focuses on Excel integration.

I don't think this would ever happen, if you can interface with COM object then you can operate it with the methods that have been implemented by the program developer. It is on us to use that with whichever language we want to. I am glad that functions like writecell, readcell and others related to these exist, as I think it's hard to interface with Excel, programmatically through COM object, so thanks TMW for that.

@Robert Guldi I'll take a look at question you posted tomorrow. Meanwhile, you can find some documentation on Excel here https://docs.microsoft.com/en-us/office/vba/api/overview/excel/object-model

[Adam Danz]

Sorry, couldn't resist.... 😃

Source: xkcd.com

[dpb]

_" you can find some documentation on Excel here..."

:) Indeed.

That's the problem--there's far too much documentation; the Object Model is so complex one soon has spent the entire day trying to figure out one little piece; particularly if new to trying to use ActiveX.

And, of course, VBA syntax doesn't directly translate to ActiveX, either...

I don't disagree that it "isn't TMW's job!" but it is a bugger to do anything at all quickly unless already completely familiar with and adept in the object model.

[Mario Malic]

Sadly, my hammer isn't good enough to understand these xkcd comics xD

If you look at complete object model, then yes, but we are mostly interacting with one (or few) objects so one needs to look at their documentation. Luckily their methods and properties are described meaningfully and it can be easy to narrow down the search. What would be cool is to be able to track the existing methods, properties and their values in the variable editor, like uiinspect does with the java components.

[dpb]

Maybe you've found " their methods and properties are described meaningfully and it can be easy to narrow down the search" to be the case, but my experience has not been so fortunate. Maybe you have enough familiarity with the object model that you can figure out how to make a Range or Cell or whatever reference to whatever it is you're looking for work, but I'm continually fight the battle of just how to get to the property of interest when it's in some collection of somethings underneath the cell or range or whatever...

I spent all day yesterday in pursuit of how to set borders programmatically and came up empty -- stuff that is documented just doesn't work or I can't figure out how to use it--another frustration I discovered raised the following Q?

https://www.mathworks.com/matlabcentral/answers/844695-enumeration-fails-with-get-set-arguments?s_tid=srchtitle

and the follow-ons to the template code I used as starting point at

https://www.mathworks.com/matlabcentral/answers/585908-excel-formatting-using-matlab#comment_1554985

If one tries to use the Xl enumerated constants to set the border line styles, Excel barfs...but if you set a given border interactively and then try retrieve the value of the constant from the saved workbook, it looks to be that same value. Whassup with that!!!??? I finally gave up in frustration as has been my luck about as much as not for anything outside just writing data.

If you can make heads nor tails of the Borders collection documentation, you're a better maven than I...I have no klew from it how it is constructed and what it means to get a Count == 6 when the doc says it is a collection of four??? And the enumerations go to 15 or so??? Makes zero sense to me.

>> excel=actxserver('Excel.Application');
>> wbk=excel.Workbooks.Open(fullfile(cd,'Testing.xlsx'));
>> wksht=Excel_utils.sheetReference(excel,1)
wksht =
	Interface.000208D8_0000_0000_C000_000000000046
K>> get(sheetReference.Range("A1").Borders,'LineStyle')
ans =
   NaN
K>> get(sheetReference.Range("D2").Borders,'LineStyle')
ans =
   NaN
K>> nBorders=borders.Count
nBorders =
     6
K>> get(borders)
     Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
         Creator: 'xlCreatorCode'
          Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
           Color: 0
      ColorIndex: -4142
           Count: 6
       LineStyle: NaN
           Value: NaN
          Weight: NaN
      ThemeColor: NaN
    TintAndShade: NaN
K>> 
K>> borders
borders =
	Interface.00020855_0000_0000_C000_000000000046
K>> borders.Count
ans =
     6
K>> borders(1)
ans =
	Interface.00020855_0000_0000_C000_000000000046
K>> borders(2)
Index exceeds the number of array elements (1). 
K>> 
% OK, so it's not an array of borders of nCount
K>> for i=1:nBorders,get(b,'item',i),end
ans =
	Interface.00020854_0000_0000_C000_000000000046
ans =
	Interface.00020854_0000_0000_C000_000000000046
ans =
	Interface.00020854_0000_0000_C000_000000000046
ans =
	Interface.00020854_0000_0000_C000_000000000046
ans =
	Interface.00020854_0000_0000_C000_000000000046
ans =
	Interface.00020854_0000_0000_C000_000000000046
K>> 
% OK, so whatever an Item is, there are six of those???
K>> for i=1:nBorders,b=get(borders,'item',i),get(b),end
b =
	Interface.00020854_0000_0000_C000_000000000046
     Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
         Creator: 'xlCreatorCode'
          Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
           Color: 0
      ColorIndex: -4142
       LineStyle: -4142
          Weight: 2
      ThemeColor: NaN
    TintAndShade: NaN
b =
	Interface.00020854_0000_0000_C000_000000000046
     Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
         Creator: 'xlCreatorCode'
          Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
           Color: 0
      ColorIndex: -4142
       LineStyle: -4142
          Weight: 2
      ThemeColor: NaN
    TintAndShade: NaN
b =
	Interface.00020854_0000_0000_C000_000000000046
     Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
         Creator: 'xlCreatorCode'
          Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
           Color: 0
      ColorIndex: -4105
       LineStyle: 4
          Weight: -4138
      ThemeColor: 'Invoke Error, Dispatch Exception: The parameter is incorrect.←↵'
    TintAndShade: 0
b =
	Interface.00020854_0000_0000_C000_000000000046
     Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
         Creator: 'xlCreatorCode'
          Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
           Color: 0
      ColorIndex: -4105
       LineStyle: -4119
          Weight: 4
      ThemeColor: 'Invoke Error, Dispatch Exception: The parameter is incorrect.←↵'
    TintAndShade: 0
b =
	Interface.00020854_0000_0000_C000_000000000046
     Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
         Creator: 'xlCreatorCode'
          Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
           Color: 0
      ColorIndex: -4142
       LineStyle: -4142
          Weight: 2
      ThemeColor: NaN
    TintAndShade: NaN
b =
	Interface.00020854_0000_0000_C000_000000000046
     Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
         Creator: 'xlCreatorCode'
          Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
           Color: 0
      ColorIndex: -4142
       LineStyle: -4142
          Weight: 2
      ThemeColor: NaN
    TintAndShade: NaN
K>> 
% But some show an error, but no indication of which parameter it thinks is
% incorrect...
% The Xlborder enumeration for bottom line is 9, though, which is >6, but
% it also returns something that is, the value I tried to write of -4119
% which is supposed to be a double line by the enumerations values in the
% documentation???
K>> b=get(borders,'item',9)
b =
	Interface.00020854_0000_0000_C000_000000000046
K>> get(b)
     Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
         Creator: 'xlCreatorCode'
          Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
           Color: 0
      ColorIndex: -4105
       LineStyle: -4119
          Weight: 4
      ThemeColor: 'Invoke Error, Dispatch Exception: The parameter is incorrect.←↵'
    TintAndShade: 0
K>> set(b)
ans = 
  struct with fields:

     Application: {}
         Creator: {'xlCreatorCode'}
          Parent: {}
           Color: {}
      ColorIndex: {}
       LineStyle: {}
          Weight: {}
      ThemeColor: {}
    TintAndShade: {}
K>> 
K>> get(b)
     Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
         Creator: 'xlCreatorCode'
          Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
           Color: 0
      ColorIndex: -4105
       LineStyle: 1
          Weight: -4138
      ThemeColor: 'Invoke Error, Dispatch Exception: The parameter is incorrect.←↵'
    TintAndShade: 0
K>> set(b,'ThemeColor',1)
K>> get(b)
     Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
         Creator: 'xlCreatorCode'
          Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
           Color: 16777215
      ColorIndex: 2
       LineStyle: 1
          Weight: -4138
      ThemeColor: 1
    TintAndShade: 0
K>> set(b,'LineStyle',-4119)
K>> get(b)
     Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
         Creator: 'xlCreatorCode'
          Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
           Color: 16777215
      ColorIndex: 2
       LineStyle: -4119
          Weight: 4
      ThemeColor: 1
    TintAndShade: 0
K>> b=get(borders,'item',5)
b =
	Interface.00020854_0000_0000_C000_000000000046
K>> get(b)
     Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
         Creator: 'xlCreatorCode'
          Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
           Color: 0
      ColorIndex: -4142
       LineStyle: -4142
          Weight: 2
      ThemeColor: NaN
    TintAndShade: NaN
K>> set(b,'LineStyle',-4119)
K>> get(b)
     Application: [1×1 Interface.000208D5_0000_0000_C000_000000000046]
         Creator: 'xlCreatorCode'
          Parent: [1×1 Interface.00020846_0000_0000_C000_000000000046]
           Color: 0
      ColorIndex: -4105
       LineStyle: -4119
          Weight: 4
      ThemeColor: 'Invoke Error, Dispatch Exception: The parameter is incorrect.←↵'
    TintAndShade: 0
K>> 
K>> sheetReference.Parent
ans =
	Interface.000208DA_0000_0000_C000_000000000046
K>> wbk=ans;
K>> wbk.Name
ans =
    'Testing.xlsx'
K>> wbk.Save
K>> wbk.Close
>> delete(excel)
>> clear('excel')

Part of extended debug session trying to figure out how to set a double line -- no success whatever; any of the values in the enumeration other than continuous (3) or none (-4142) error but if read a cell which manually set to have the desired linestyle, it returns the same value.

I've no explanation for why nor found any way in which can make the desired change programmatically.

One can't write VBA expressions that do assignments in ActiveX syntax nor are the values actually passed for the enumerated constants visible in how they're passed other than as the decimal values shown in the documentation, so if they don't work, one's pretty-much at an impasse.

That's been my frustration on almost everything I've ever tried -- it simply is not a 1:1 translation; eventually one usually can find someone on one of the Excel newsgroups or code search that has stumbled over same/similar problem and find the magic incarnation, but I've not found anything yet to get around this one.

Sorry for the rant...but was pretty-much a lost day. :(

[Mario Malic]

I hope I understood what you wanted to do.

This line is not good.

thisBorder=get(borders, 'Item', border);

Try this

theCell=sheetReference.Range(ranges{i});
borders=get(theCell, 'Borders'); % borders return the borders that are bounded by range Items(1:4), where 5:6 are diagonals
set(borders,'LineStyle', style,'Weight',weight);

There are some weird things of course which are misisng from the documentation. borders.Count returns 6, borders.Items go to 12 (probably when selection is 2D). If you set borders.LineStyle to some value, 1-4 are set.

[Paul]

@dpb

Where in your code is the variable borders defined, as used in this line?

nBorders=borders.Count

@Mario Malic

Where did you see this line?

thisBorder=get(borders, 'Item', border);

[dpb]

Paul/Mario -- thanks for taking an interest... :) I really didn't expect anybody to delve into this; I was just frustrated and ranting...

Here's the little testing routine I wrote thinking if I manually set some borders in a spreadsheet I could then retrieve what was in the cells and use that to figure out what should write -- needless, to say, I left more confused than I came--

function GetBorderInfo(sheetReference, ranges)
% return borders collection info  for range given
  if ischar(ranges), ranges=cellstr(ranges); end
  for i=1:numel(ranges)
    try
      theCell=sheetReference.Range(ranges{i});
      borders=get(theCell, 'Borders');
      nBorders=boarders.Count;
      
      thisBorder=get(borders, 'Item', border);
      set(thisBorder,'LineStyle', style,'Weight',weight);
    catch ME
      fprintf('Error in function SetBorder.\nError Message:\n%s\n', ME.message)
      %warning('Error in function SetBorder.\n%s', ME.message)
    end
  end
end

I opened the ActiveX session first, then used

GetBorderInfo(wksht,'D2')

and then poked around in the debugger trying to figure out what was what.

I did recognize the problem line so set the breakpoint at the beginning of the

      thisBorder=get(borders, 'Item', border);

line and then did all my poking around in the debugger. I just got so frustrated I never tried to fix the code but just gave it up as lost cause and wasted day.

If you look at the doc for enumerations

https://docs.microsoft.com/en-us/office/vba/api/excel.xllinestyle

the values are

Name 	        Value 	Description
xlContinuous 	    1 	Continuous line.
xlDash 	        -4115 	Dashed line.
xlDashDot 	    4 	Alternating dashes and dots.
xlDashDotDot 	    5 	Dash followed by two dots.
xlDot 	        -4118 	Dotted line.
xlDouble 	-4119 	Double line.
xlLineStyleNone -4142 	No line.
xlSlantDashDot 	   13 	Slanted dashes.

but any value I tried to write other than 1 or -4142 returned the error and didn't set the line style desired, but I did, in fact see the value -4119 returned from poking around after I had manually set and save the cell border in cell D2 of the test workbook.

I guess the thing about " If you set borders.LineStyle to some value, 1-4 are set" explains why the enumerations begin at 5 although if that is documented anywhere, I've not been able to find it, nor do any examples mention the fact.

So, what I wanted was a bottom double line of either thin or medium width...but tried to write a generic routine to be able to set any as desired.

It worked/works for a continuous line, but nothing else...gets what appears a random selection of continusous or various dotted/dashed, etc., but never a double to be seen no matter what I tried to write.

It's also not documented what the expected type of the values passed is to be; afaict by symptoms, MATLAB set() converts any native numeric to whatever internally although as the other Q? I posted notes (to which there's been no response so far) the MATLAB enumeration type seems to actually pass some object, not the equivalent value as value as one would expect an enumeration should/would do. That's just rude!!!

Anyways, any further light you can shed would be greatly appreciated, but I remain in the camp believing that writing ActiveX to interact with Excel is worse than having root canal without anesthesia...

[dpb]

Oh, here's the generic routine I tried to write, taken from Image Analyst's utilities package that illustrated some specific choices rather than being general. You'll see the kludge to get around the MATLAB enumeration problem--I had built those classes as a user aid to be able to see the constants instead of having to look them up or have in comments or bury magic constants in source code. That proved to be, so far, an almost pointless exercise, too.

function SetBorder(sheetReference, ranges, border, weight, style)
% borders is a collections of all cell borders. Must set each.
%
% my_border = get(borders, 'Item', XlBordersIndex);
% set(my_border, 'ColorIndex', 3);
% set(my_border, 'LineStyle', 9);
%
% Excel XlBordersIndex Enumeration
%  5 - xlDiagonalDown     UL diagonal down to LR
%  6 - xlDiagonalUp       LL diagonal up to UR
%  7 - xlEdgeLeft         Border at the left edge of the range.leftmost only
%  8 - xlEdgeTop          Border at the top of the range.
%  9 - xlEdgeBottom       Border at the bottom of the range.
% 10 - xlEdgeRight        Border at the right edge of the range.
% 11 - xlInsideVertical 	Vertical borders for all the cells in the range except borders on the outside of the range.
% 12 - xlInsideHorizontal Horizontal borders for all cells in the range except borders on the outside of the range.
%
% Excel XlBorderWeight Enumeration
% Specifies the weight of the border around a range.
% Name          Value     Description
% xlHairline    1         Hairline (thinnest border).
% xlMedium      -4138     Medium.
% xlThick       4         Thick (widest border).
% xlThin        2         Thin.
% 
% Excel XlLineStyle Enumeration
% Specifies the line style for the border.
% Name            Value 	Description
% xlContinuous    1       Continuous line.
% xlDash          -4115 	Dashed line.
% xlDashDot       4       Alternating dashes and dots.
% xlDashDotDot    5       Dash followed by two dots.
% xlDot           -4118 	Dotted line.
% xlDouble        -4119 	Double line.
% xlLineStyleNone -4142 	No line. (Alias xlNone)
% xlSlantDashDot 	13      Slanted dashes.

  if ischar(ranges), ranges=cellstr(ranges); end
  if isenum(border), border=int32(border); end
  if isenum(weight), weight=int32(weight); end
  if isenum(style), style=int32(style); end
  for i=1:numel(ranges)
    try
      theCell=sheetReference.Range(ranges{i});
      borders=get(theCell, 'Borders');
      thisBorder=get(borders, 'Item', border);
      set(thisBorder,'LineStyle', style,'Weight',weight);
    catch ME
      fprintf('Error in function SetBorder.\nError Message:\n%s\n', ME.message)
      %warning('Error in function SetBorder.\n%s', ME.message)
    end
  end
end

It also has the aforementioned bizarre behavior that the warning line is totally ineffective -- I just thought about the fact this routine is still a static method in a class having kept the organization IA had posted. Haven't tested whether moving it into its own m-file would fix that problem--good guess it might I think. Although I couldn't find anything to that end in the MATLAB doc, either.

[Paul]

I'm quite sure I won't be able to offer any help. I was just curious. But I feel your pain having dabbled with doing much simpler mucking around in PowerPoint from Matlab.

[Mario Malic]

borders.Item(1).LineStyle = -4119 will give you double line on one border, I think the right one. Bottom border is one of Item between 2 and 4 so check those.

[dpb]

So whassup w/

%  9 - xlEdgeBottom       Border at the bottom of the range

in the doc enumerations and nothing listed for 1 thru 4???

This still makes no sense to me whatsoever...

I'm in the middle of another debug session at moment so will check again but I tried a loop of 1:N before and still had no joy I'm pretty sure...

Thanks for the feedback; will report.

[dpb]

OK, thanks for the hint -- I had thought that

      borders=get(theCell, 'Borders');
      theBorder=get(borders, 'Item', border);
      set(theBorder,'LineStyle', style,'Weight',weight);

would have returned the wanted border Item object, but apparently not...I still don't fully understand that, but the following does work....

      theBorder=borders.Item(whichBorder);
      set(theBorder,'LineStyle', style,'Weight',weight);

although one apparently can't write

      set(borders.Item(whichBorder),'LineStyle', style,'Weight',weight);

in typical MATLAB syntax fashion.

function SetBorder(sheetReference, ranges, whichBorder, weight, style)
  if style==XlLineStyle.xlDouble, weight=XlBorderWeight.xlThick; end % only combination that works
  if ischar(ranges), ranges=cellstr(ranges); end
  if isenum(whichBorder), whichBorder=int32(whichBorder); end
  if isenum(weight), weight=int32(weight); end
  if isenum(style), style=int32(style); end
  for i=1:numel(ranges)
    try
      range=sheetReference.Range(ranges{i});
      borders=get(range, 'Borders');
      theBorder=borders.Item(whichBorder);
      set(theBorder,'LineStyle', style,'Weight',weight);
    catch ME
      fprintf('Error in function SetBorder.\nError Message:\n%s\n', ME.message)
      %warning('Error in function SetBorder.\n%s', ME.message)
    end
  end
end

Experimentation also revealed that one can't use any combination of the enumerations (which limitation is also NOT documented). Only with the "Thick" weight is a double-line rendered; all the others are a single line of medium weight.

All kinds of things aren't documented and left for only trial and error to uncover...

The above could be cleaned up and forget about trying to use enumerations, but it's expedient for the moment and there are far more important issues to fix rather than continuing to throw time down the rat hole...

Thanks, you did get me pointed to what can be made to work--muchly appreciated!!

Answer 22

[cui]

Single and multi-objective vision tracking algorithms:

KCF, GOTURN, FairMOT, deepSort ,... are some of the best and successful algorithms in recent years, but unfortunately, why is there no such implementation in the "sensor fusion and tracking toolbox", "deeplearning toolbox" and "computer vision toolbox"?

loss functions for time series prediction:

The current deep learning toolbox is missing loss functions for time series prediction, e.g. "DTW, Soft-DTW, DILATE" loss, and although the signal processing toolbox has a dtw function, it does not support dlarray type data input?

Video reading and processing

The latest version of R2021a only supports fileDatastore/videoReader etc. to load various video datasets, and the read IO bottleneck is very inefficient and lacks a decord-like library for efficient video processing

Bayesian Deep Learning Convolution Network(BDL),Bayesian Neural Network(BNN)

https://www.mathworks.com/matlabcentral/answers/886734-bayesian-deep-learning-convolution-network-bdl

https://www.mathworks.com/matlabcentral/answers/376-bayesian-neural-network#answer_373796

Binary Neural Networks

Does current matlab 2021a deeplearning toolbox support Binary Deep Neural Networks (BNN)? How is it trained and compressed? If not, will future versions integrate this feature? thanks

reference:

时间序列预测损失函数 DTW, Soft-DTW, DILATE

[Adam Danz]

@cui your suggestions are imaginative, specific, and I'm sure it's valuable to the company to see what users would like. It would be even more helpful to include a comparison to existing features that come close to achieving your suggestions. I'm not an employee of MathWorks nor am I speaking for them, this is just my opinion.

For example, there are already object tracking algorithms available in Matlab (see computer vision toolbox). At the current pace of technology, algorithms are created much faster than the pace of development, especially for companies like MathWorks that must thoroughly test for backward compatibility and redundancy. This is where the File Exchange often comes in handy to fill in the gaps. By listing every possible missing algorithm, the SNR of focal points for future releases becomes weaker. That could be strengthened by including arguments for why a new feature is needed rather than just listing missing features.

[cui]

@Adam Danz,Thank you for your useful advice！This is another indication that matlab is very good, not the latest, but the most robust and reliable algorithm integration!

Answer 23

[Image Analyst]

I'd like questdlg() to be able to take more than 3 buttons.

Often I need 4 buttons, like "Yes", "No", "Cancel", and "All", like if I'm looping over files in a listbox asking if you want to delete them or not. Why can't questdlg() take any number of arguments, much like other functions like ones(), and just put up the buttons in a dialog box in an array? The last argument could be the button that you'd like for the default (as it is now).

I know I can still use menu() but it's deprecated in favor of listdlg() and I'd rather have buttons than a listbox - it's just more intuitive.

Answer 24

[Walter Roberson]

Currently evalin() accepts a context argument (such as 'caller' or 'base' or symengine()), and a character vector that is the command to be executed.

This runs into the same horrors as using eval() .

It is difficult to convince people to give up using eval(); they tend to think that their particular use case makes it necessary (it rarely is!). And the cause is Not Helped At All when we have to say "well, it is true that the closely-related "evalin() is needed sometimes"

It would therefore help if there was a way to do something like evalin() but with a function handle. For example,

evalin('caller', 'whos')

might become something like

evalfcnin('caller', @whos)

To be honest, I do not know how this would work in practice, considering that functions need their own workspace to execute in. Maybe some of the technology behind shared variables and nested functions could be used, so that the function could have read/write access to the designated function workspace and yet still be able to have its own private variables.

Sometimes I want to be able to get a look at persistent variables in a function that is not the caller; I have never found a hint that is possible. But the existence of persistent variables that go away when you "clear" the function, implies that each parsed function already has some kind of workspace associated with it. And sometimes it seems to me that it would be useful if you were able to get a referene to that workspace and tell a function to execute using that workspace. Something that might look like

evalinWorkSpace( matlab.workspace.getinstance.caller, @whos)

Having workspaces as accessible objects leads to some interesting possibilities about saving and restoring state, such as for the purpose of recovering from power failures; it also heads towards possibilities such as co-routines.

Answer 25

[Valeri Aronov]

Your TypicalX option for fminunc() (and others?) should be extended beyond usage for gradient evaluation only.

I have used normalized variables for optimization for a long time. By default, I do start with [1,1,...,1] as you do, but for gradient evaluation only. There are many areas where a spread of variables is vast (radiolectronics: kiloohms to nanofarads).