Researchers find 39 unreported sources of major pollution: NASA

2016-06-02 17:17:37

Researchers in the United States and Canada have located 39 unreported sources of major pollution using a new satellite-based method, the U.S. National Aeronautics and Space Administration said.The unreported sources of toxic sulfur dioxide emissions are clusters of coal-burning power plants, smelters and oil and gas operations in the Middle East, Mexico and Russia that were found in an analysis of satellite data from 2005 to 2014, NASA said in a statement on Wednesday. The analysis also found that the satellite-based estimates of the emissions were two or three times higher than those reported from known sources in those regions, NASA said. Environment and Climate Change Canada atmospheric scientist Chris McLinden said in a statement that the unreported and underreported sources accounted for about 12 percent of all human-made emissions of sulfur dioxide. The discrepancy could have "a large impact on regional air quality," said McLinden, the lead author of the study published in Nature Geosciences. A new computer program and improvements in processing raw satellite observations helped researchers at NASA; the University of Maryland, College Park; Dalhousie University in Halifax, Nova Scotia; and Environment and Climate Change Canada detect the pollution, according to the U.S. space agency. The researchers also located 75 natural sources of sulfur dioxide in the form of non-erupting volcanoes that are slowly leaking the toxic gas. Although the sites are not necessarily unknown, many volcanoes are in remote locations and not monitored, so the satellite-based data is the first to provide regular annual information on these volcanic emissions, NASA said. (Reporting by Brendan O'Brien in Milwaukee; Editing by Scott Malone and Lisa Von Ahn)

Message Durability in ActiveMQ 5.X

2016-05-26 18:13:04

I get asked quite a bit to explain the basics of how ActiveMQ works with respect to how it stores messages (or doesn’t in some cases). Here’s the high-level explanation of it. Note, the context is within JMS. If you use ActiveMQ’s non-JMS clients (e.g., STOMP, AMQP, MQTT, etc.) then the behavior may be different in some cases.ActiveMQThe JMS durability guarantees are pretty strong in terms of not losing messages that are marked “persistent.” Let’s see how that applies for ActiveMQ.TopicsTopics are a broadcast mechanism. They allow us to implement publish-subscribe semantics in JMS land. However, what happens if we mark a message “persistent” and there are no subscribers? In any normal broadcast (e.g., I go downtown and start shouting about the awesomeness of ActiveMQ), if there are no subscribers (it’s 3am and there’s nobody around to hear me…. must’ve been a good night out if I'm out at 3am) then what happens? Nothing. Nobody hears it. And we move on. ActiveMQ doesn’t do anything with the message if you publish it (persistent or not persistent) and there are no subscribers (no live subscribers and no durable subscribers).ActiveMQ will only store the message if there are durable subscribers (active or inactive). For an inactive durable subscription, ActiveMQ will store messages marked “persistent” into a non-volatile store and wait for a subscriber to rejoin the subscription. At that point it will try to deliver messages.QueuesFor queues, ActiveMQ treats “persistent” messages with a simple default protocol. We basically block the main producer thread and wait for confirmation that the broker has actually gotten the message:Producer:Producer sends messageProducer blocks, waits for ACK from brokerProducer continues on if successful ACKRetries if NACK or timeout or failoverBroker:receives messagestores message to disksends back ACKFor “non-persistent” sends, the flow is different. We send in a “fire and forget” mode. The main producer thread does not get blocked and any ACK or other response happens asynchronously on the ActiveMQ Connection Transport thread:Producer sends messageProducer continues on with its thread and does not blockProducer eventually gets ACK on a separate thread than the main producer threadTransacted sends?We can increase performance of sends to the broker by batching up multiple messages to send at once. This utilizes the network as well as the broker storage more effectively. There’s an important distinction you must be aware of when sending transacted. The opening of the TX session and the closing of it (rollback/commit) are all synchronous interactions with the broker, however, the sends for each individual message during the TX window are all sent asynchronously. This is okay if everything works out because the broker batches these messages up. But what happens if there are transport errors? Or the broker runs out of space to save these messages?We need to set an ExceptionListener to watch for errors during these sends. We also need to (or should) set a client side sending “producer window” to allow us to enforce producer flow control when the broker runs out of resources. See ActiveMQ producer flow control for more.Changing the defaultsThe interesting settings on the producer that can change these behaviors:useAsyncSend - always wait for ACKs asynchronously, even in persistent sends and commitsalwaysSyncSend – force all sends (non-persistent or transactional sends included) to always wait for ACK from the brokerUsing the defaults are generally what folks want.StorageFor production usage of ActiveMQ, I recommend the shared-storage approach at the moment. In this case, we need to be aware of what’s happening at the storage layer to understand ActiveMQ’s guarantees.ActiveMQ by default will implement JMS durability requirements which basically states messages that get stored must survive crashes. For this, we by default will do a “fsync” on the filesystem. Now what happens on each system will be dependent on what OS, network, storage controller, storage devices, etc you use. This is the same you’d expect for any type of database that needs to persistently store messages and is not ActiveMQ-specific, per se.When we write to the ActiveMQ transaction journal we need to ask the OperatingSystem to flush the journal to disk with a call to fsync. Basically what happens is we force the OS to write back the page-file cache it uses to cache file changes to the storage medium. It also encourages the storage medium to do what it needs to do (depends on implementation) to “store” the data to disk:Some storage controllers have their own cache that needs to be flushed. The disk drives have their own caches, etc. Some of these caches are backed by battery and may write-back at their own time intervals, etc. For you to understand the durability of your messages running through ActiveMQ, you should understand the guarantees of your storage layer.ConsumersFinally the last piece of the puzzle is how we deliver/dispatch messages to consumers and how they acknowledge. The ActiveMQ JMS libraries handle all of this for you, so you don’t need to worry about whether or not you’re going to lose messages.Messages get dispatched to consumers up to a certain “prefetch” buffer that lives on the consumer. This helps speed up message processing by having an available cache of messages on the consumer ready to process and then refill this cache as the consumer consumes them. In ActiveMQ these prefetched messages are denoted as “in flight” in the console. At this point it’s up to the consumer to process these messages and ACK them. (this will depend on the ack modes… the default of auto ack will send the ACK as the consumer gets the message.. for more important message processing you may wish to use “client” ack where the client explicitly says when to ack the message, i.e., after it’s completed some processing).If the consumer fails for some reason, any of the non-ack’d messages will be redelivered to another consumer (if available) and follow the same processing as above. The broker will not remove the message from its indexes until it gets an ACK. So this includes failures at both the consumer and network level. If there are errors at either of these levels even after a consumer as “successfully processed” (note, this is very use-case specific what “successfully processed” means), and the broker does not get the ack, then it’s possible the broker will re-send the message. In this case you could end up with duplicates on the consumer side and will probably want to implement an idempotent consumer. For scaling up messaging producers/consumers, you’ll want to have idempotent consumers in place anyway.Last thing to note: JMS DOES NOT GUARANTEE ONCE AND ONLY ONCE PROCESSING of a message without the use of XA transactions. JMS guarantees once and only once delivery insofar it can mark messages as being “redelivered” and have the consumer check that, but the consumer is responsible for how many times it should be allowed to process (or filter out with idempotent consumer).

Spring LDAP 2.1.0 Released

2016-05-18 19:13:05

On behalf of the community, I’m pleased to announce the release of Spring LDAP 2.1.0.RELEASE. The highlights of this release include:#380 - Support for Spring Data Hopper#384 - Early support for Spring IO Platform 2.1#351 - Support for commons-pool2#370 - Support property placeholders in XML Namespace#392 - Document Testing SupportMigrated from JIRA to GitHub IssuesAdded Gitter ChatFor complete details of 2.1 refer to the changelog for 2.1.0.RC1 and 2.1.0.RELEASEIf you have feedback on this release, I encourage you to reach out via StackOverflow, GitHub Issues, or via the comments section. You can also ping me @rob_winch or Joe (our latest full-time Spring Security team member) @joe_grandja on Twitter.

Senate committee questions Facebook over news selection

2016-05-11 05:24:06

A U.S. Senate committee launched an inquiry on Tuesday into how social media website Facebook selects its news stories after a report that company employees blocked news about conservative issues from its "trending" list.The Senate Committee on Commerce, Science and Transportation asked Facebook Chairman and Chief Executive Mark Zuckerberg in a letter to answer questions about the company's news curation practices and its trending topics section.The investigation comes after Gizmodo reported on Monday that a former Facebook employee claimed workers "routinely suppressed news stories of interest to conservative readers," while "artificially" adding other stories to the trending list.U.S. Senator John Thune, the chairman of the committee, told reporters Tuesday his primary concern was that Facebook may be engaging in deceptive behavior if employees meddled with what trending news was displayed.“If you have a stated policy, which your followers or your audience knows to be the case, that you use an objective algorithm for trending topics -- you better follow that policy,” Thune said. “It’s a matter of transparency and honesty and there shouldn’t be any attempt to mislead the American public.”The letter to Facebook includes requests for information on the organizational structure for the "Trending Topics feature." Adam Jentleson, deputy chief of staff to Democratic Senator Harry Reid, balked at the request in a statement provided to Reuters."The Republican Senate refuses to hold hearings on [Supreme Court nominee] Judge [Merrick] Garland, refuses to fund the president’s request for Zika aid and takes the most days off of any Senate since 1956, but thinks Facebook hearings are a matter of urgent national interest," Jentleson said.A Facebook spokesman said it had received Thune's request for more information about how "Trending Topics" works. "As we investigate, we will also keep reviewing our operational practices around Trending Topics — and if we find they are inadequate, we will take immediate steps to fix them," the spokesperson said.Tom Stocky, the vice president of search at Facebook, responded to the allegations Monday night in a lengthy post published to the social media site saying there are "strict guidelines" for trending topic reviewers who "are required to accept topics that reflect real world events." He added that those guidelines are under "constant review" and that his team would "continue to look for improvements." Katie Drummond, the editor-in-chief of Gizmodo, called her publication's story "accurate" in a statement released to Reuters Tuesday.Gizmodo's report alarmed several social media users, with some conservatives in particular criticizing Facebook for alleged bias."'If a Conservative Speaks - and Facebook Censors Him - Does He Make a Sound?'" Wisconsin Governor Scott Walker (@ScottWalker) wrote on Twitter Tuesday, with a link to a National Review story that detailed the allegations against Facebook. (Reporting By Amy Tennery; additional reporting by Dustin Volz in Washington; Editing by Alan Crosby)

Diving into Visual Studio 2015 (Day #1) : Code Assistance

2016-05-10 02:10:07

In this series, I’ll cover how development with Visual Studio 2015 can increase your productivity and enable you to write cleaner and more optimized code.Download - 1.2 MBIntroductionI have always been a great admirer of Visual Studio IDE (Interactive Development Environment). Visual Studio has proved to be the best IDE for me and I use it for almost all my coding as well as debugging work. My love for the IDE has forced me to start a series of articles to explain what Visual Studio 2015 now offers to a developer in terms of cross-platform development, cloud-based development, code assistance, refactoring, debugging, and more. The power of Visual Studio is not only limited to development and coding but it offers a one-stop solution to all the requirements needed during coding, development, code analysis, or deployment. I’ll use Visual Studio Enterprise 2015 throughout the series and explain how one can leverage Visual Studio to be more productive. In this section of the series, I’ll cover how development with Visual Studio 2015 can increase your productivity and enable you to write cleaner and more optimized code.Code AssistanceIn earlier versions of Visual Studio, you must have seen that whenever you write a buggy code, the code editor provides suggestion with the help of a tool tip. This feature has improved a lot and is shown as a light bulb icon in the Visual Studio code editor. This option provides you with the real-time suggestions while coding in Visual Studio code editor to improve code quality or fix the coding issues. It helps you in identifying syntax errors, provides useful code hints and assists you with static code analysis. I am using a sample code to explain the enhancements, for that I have created a console application in my Visual Studio and named it VisualStudio2015ConsoleApplication.Syntax Error SuggestionsSuppose there is syntax error in your code like I purposely did in the below image:The light bulb icon immediately shows up when you click your mouse over the erroneous variable having a red line and displays an issue summary, an error code with a link to documentation. It also displays a possible list of code fixes and refactorings. In the above example, I am writing an add method, taking two parameters a and b, but I am trying to return a result as a+bh. Now since "bh" is not declared anywhere in the method or passed as a parameter, the light bulb icon shows up and provides certain possible options or suggestions about how this variable can be taken care of. It suggests generating a variable named "bh", create a field or property as well.If you hover on the error line you’ll be shown a light bulb icon showing error and potential fixes.Note that you can also use Ctrl+. to see the error using your keyboard. If you click on Show potential fixes, you’ll get the same options as shown in the first image. Alternatively, if by any chance you doubt the light bulb icon and build your console application, you’ll again be shown the same error as follows:The syntax error assistance displays a light bulb icon, a description of the error, and a link to show possible fixes as well. When you click on the error code i.e. CS0103 you’ll be redirected to the documentation of the error code. It also offers to preview changes once you go for any of the suggestions provided by the light bulb icon. So if I click on Preview changes it shows me the preview of the option that I have chosen as shown below.Now we don’t have to go to the code and explicitly define that variable. Just click on Apply button and Visual Studio takes care of everything. Therefore, the first option that I chose is now reflected in my code.I remember that I used to do these on the fly modifications to improve productivity using ReSharper. We saw that we got the same error on compiling the application which proves that light bulb icon’s code suggestion can help us write error free code before even compiling the application and getting to know about the actual error after a compile. Therefore, we don’t have to wait to compile the application to know about compile time error. You can test different scenarios to explore the syntax error suggestions given by the light bulb icon in our day to day programming.Code SuggestionsLet’s take another scenario. Suppose I define an interface named ICalculator and add a class named Calculator and inherit the calculator.cs class from that interface.Interface1: interface ICalculator 2: { 3: int Add(int a, int b); 4: int Subtract(int a, int b); 5: int Multiply(int a, int b); 6: float Divide(float a, float b); 7: } Class1: public class Calculator : ICalculator 2: { 3: } You’ll see that there will be a red error line under the ICalculator interface named in calculator class. You can get to see the light bulb icon in the same way as shown in previous example i.e. hover or click on error. Here you’ll see that light buld icon is assisting us with some additional conceptual information, like that the interface that we are using contains several methods that needs to be implemented in the Calculator class.Therefore, we see that light bulb not only assists us in finding syntax errors but also suggests a conceptual or logical resolution of mistakes we make in our programming. When you show on "Show potential fixes link" it will show all the possible fixes for this error in a detailed user-friendly manner with an option to resolve and fix it with preview as well.In the above image, you can see that the code assistance is providing an option to either implicitly and explicitly implement an interface ICalculator, and if we analyse we can clearly say that these are the only possible options that a developer may opt for in this scenario. Moreover, it shows the error does link referring to its description. If you choose the first option and choose preview changes link, you’ll see the following preview and you can choose to apply that if it is what you need.So click apply and we get the following class with all the interface methods having default implementations. 1: public class Calculator : ICalculator 2: { 3: public int Add(int a, int b) 4: { 5: throw new NotImplementedException(); 6: } 7: 8: public float Divide(float a, float b) 9: { 10: throw new NotImplementedException(); 11: } 12: 13: public int Multiply(int a, int b) 14: { 15: throw new NotImplementedException(); 16: } 17: 18: public int Subtract(int a, int b) 19: { 20: throw new NotImplementedException(); 21: } 22: } Likewise, the light bulb icon provides numerous code suggestion options while development and coding following which we can increase the productivity of writing code without unnecessarily compiling the application and writing the code manually.Refactoring SuggestionsThe light bulb icon is not only limited to code suggestions and syntax error suggestions but also comes with a great capability of refactoring techniques. When I added the calculator class, the class was added with few default namespaces as shown below.In the above scenario as we can see there are a few namespaces added by default in the class that are currently not used. When you hover the mouse over those namespaces, the light bulb icon shows up with some refactoring suggestions as shown below.The above image shows the suggestion of Lightbulb icon asking to remove "unnecessary usings." We see here that Visual Studio is smart enough to know what refactoring is required in the code and accordingly can suggest the developer to optimize the code. If you apply the suggestion it will remove the unnecessary "usings" from your code. You can also select to fix all occurrences of this issue in the current document, the project, or the solution. If we just want to make this local change, we can select Remove Unnecessary Usings here, and the unused usings are removed.Quick Suggestions and RefactoringNow when I go to my Calculator.cs class and define the Add method as follows.1: public int Add(int a, int b) 2: { 3: int c = a + b; 4: return c; 5: } It is the correct way of defining an add method, but on second thought, what if I want to refactor or optimize this method? Visual Studio 2015 provides us with the facility to do quick refactoring of code with suggestions using an option enabled in the context menu of the editor. Just right-click on "c" and you’ll get to see an option at the top of the context menu saying "Quick Actions and Refactorings…".Note that in above code block, Visual Studio didn’t suggest anything implicitly and due to some syntax error but we have the option to choose and ask for Visual Studio’s help explicitly to know if a particular written code could be enhanced, optimized, refactored more or not. There could be cases that choosing this option too does not show any suggestion to improve code, which means your code is already refactored and optimized. But in the above-mentioned scenario, if we select the "Quick Actions and Refactorings…" option, VS gives us two options to further optimize the code.or,If we have a glance over both the options, the first says to skip using temporary variable and just return (a+b) (which is a good suggestion by the way) and the second option says to extract a method out of the code and put (a+b) in any other method and return from there. Now in these situations, it is the choice of the developer on what option he chooses. I choose the first option and apply the changes that it showed me in preview and I got following code which looks better that the earlier one.1: public int Add(int a, int b) 2: { 3: return a + b; 4: } Note that these are the small examples that I am taking to just explain the power of Visual Studio 2015. There could be complex and tricky scenarios where you may actually need a lot of help through these features.ConclusionI took few representative examples from a large set of possible scenarios where Code assistant of Visual Studio 2015 can help you. You can explore the IDE and play around to see more such situations where you feel the change from earlier versions of Visual Studio to this one. In the next section, I’ll talk about Live Static Code Analysis. For more technical articles you can reach out to my personal blog, CodeTeddy.References

Older PostNewer Post
Twitter users decry reported plan to prioritize tweets
Lockheed unit to help design quieter supersonic passenger jet: NASA
Researchers find 39 unreported sources of major pollution: NASA